[VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Please vote on the following plan for Hadoop release 2.0.5
- bug fixes encountered in current release 2.0.4-alpha
- make all API changes to allow freezing them post 2.0.5
- no new features

As discussed on @dev thread
http://s.apache.org/fs
this will allow to stabilize 2.0 branch in a short and predictable period
of time.
This enables a powerful option to have the release tested at Yahoo scale.
The plan is to follow up with 2.1.0 - the stable release.
New features can and should be added on top of the stable release once it
is out.

Hadoop by-laws:
http://hadoop.apache.org/bylaws.html

"Release Plan
Defines the timetable and actions for a release. The plan also nominates a
Release Manager.
Lazy majority of active committers"

assume nomination of a Release Manager with the plan.
It would be really good if Arun continues if this plan is adopted.
We can return to the RM topic if not.

The vote will run for 7 days until next Wed, May 8th.

Thanks,
--Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Nauroth <cn...@hortonworks.com>]

-1 (non-binding)

I want to thank everyone on the thread for taking the time to present the
potential risks and benefits so clearly.  There are valid arguments on
either side.  My -1 vote is based on the fact that maintaining an extra
branch carries a high cost, and I do not see evidence that the potential
stability benefits would outweigh that cost.

Committers already have a heavy workload, and this plan would create a
scenario where committers need to manage merges not only between trunk and
branch-2, but also to a separate branch-2.0.  This is in addition to
possible backports to branch-1 or 0.23.7.

I also do not see a definition of any specific development or testing
activity, beyond a vague sense of "bake time", that needs to happen inside
the new branch-2.1 for the new features to be considered stable.  As others
on the thread have said, we all can work more effectively if stability
concerns are documented clearly as requirements in jiras, so that we have a
measurable goal and an entry point for engineers that want to volunteer on
stabilization.

As I understand it, the features under debate are secure short-circuit
reads, snapshots, and Windows compatibility.  In all 3 cases, I've observed
multiple contributors, committers, and testers actively working towards the
goal of stability.  I cannot think of any specific deficiencies that I need
to see addressed before I would deploy the code.  (If I do think of
anything later, I'll document it in a jira, and I'm happy to help with
stabilization of any of those features.)

Of course, every new line of code carries risk.  There will always be bugs.
 In this case though, I think the proposal increases development workload
without significantly reducing the risk.

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Mon, May 13, 2013 at 9:36 AM, Robert Evans <ev...@yahoo-inc.com> wrote:

> If I have to get to the other end of 14 busses I can either do it Evel
> Knievel style and jump them all at once or I can walk from one end to the
> other, one step at a time.  I personally would rather take many different
> small steps than try to take one giant flying leap and risk missing.
>
> Also I don't think anyone said that we are going to to a major release per
> feature.  I thought the proposal was to do a 2.0 release before snapshots
> is merged in and a have snapshots go into 2.1.  I don't really want to
> open up the version numbering discussion again but I want my quote to be
> interpreted the way I intended it to be.  I don't see snapshots as a major
> change.  I see it as a step.  I just don't see a reason to keep two
> parallel lines branch-2 where we try hard to maintain compatibility, and
> trunk where is is a dumping ground that only makes it more difficult to
> move to in the future.
>
> --Bobby
>
> On 5/10/13 1:55 PM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>
> >
> >On May 9, 2013, at 10:03 PM, Uma Maheswara Rao G wrote:
> >
> >>  Adding new features really a great thing and surely each big feature
> >>can be included in one one major release as well.
> >
> >
> >Surely the one thing we have learnt over the last 4-5 years in this
> >project is that we cannot make too many major releases. Notice how long
> >hadoop-0.20/hadoop-1 lives; the same will happen with hadoop-2.x. Look at
> >Bobby's message on this thread on *-dev lists:
> >
> >>>> Up to this point we have almost successfully done this switch once,
> >>>>from 1.0 to 2.0. I have a hard time believing that we are going to do
> >>>>this again for another 5 years.
> >
> >Each major release is a *lot* of work and is, subsequently, an
> >opportunity where incompatibilities at various levels creep in - our
> >users are not well served by this.
> >
> >So, it's a  fallacy that we can and should make major releases per
> >feature.
> >
> >Arun
> >
>
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

> It's possible we're confused. Your proposal sounds like Arun should RM
> a release with a particular profile. I apologize if I assumed more
> than you intended.

Apologies accepted.
It is really hard to explain things to anybody who doesn't want to hear you.

> You have all the tools
> and authority you need to create a release. Nobody has authority to
> block you,

Wouldn't it be good to come to common ground and work on a single community
release.

> your technical input on the particular features would be most welcome.

Again as I said before it is not about technical issues, on which I spent
as much time as I could - wish I had more.
Repeating myself here:
I am not challenging any features, the implementations, or the developers.
Adding a 500 KB patch invalidates prio testing solely because it is a big
change that needs testing not only by itself but with upstream applications.
With 2.0.3 , 2.0.4 tested thoroughly and widely in many organizations and
several distributions it seems like a perfect base for the stable release.

What work for me may not work for you. What works at Twitter may not work
at Intel. What is compatible with HBase may be incompatible for Impala.
What runs on Oracle Java may break on IBM's. What passes on Centos 6.1 may
fail on Suse 11. What is optimal for 100 node cluster stalls at Yahoo
scale.
There is too many dimensions some of which we don't even know about.
Stability doesn't come by declaration or tagging. It rather happens where a
critical mass accumulates, sometimes unpredictably. We just need to
recognize the moment.

People indeed spent a lot of time expressing themselves. Means to me it is
important to them. It would be sad to see the effort wasted.

Thanks,
--Konstantin

On Mon, May 13, 2013 at 2:46 PM, Chris Douglas <cd...@apache.org> wrote:
>
> Bobby-
>
> As much as I enjoy the Evel Knievel image, your argument is not
> finding traction for lack of a visceral metaphor. It's the lack of
> detail. We're not jumping buses, we're adding features to code, where
> it's possible to be specific. If you're uncomfortable with the design,
> implementation, or test plan of a feature, then share your
> reservations. Either someone can reassure you that your issue is
> covered by existing tests, they can add new tests, or- given enough
> evidence- we can agree that the feature needs more time to bake before
> being added to the beta. If you need extra time to do this, please
> insist. Given all that's been written, I literally can't believe that
> nobody has time to do this, and it would be a lot more productive.
>
> I share your concerns about trunk, abstractly. Currently, there's
> almost nothing there that isn't in branch-2 (which makes metaphors
> like "junkyard" and "dumping ground" sound a little hysterical,
> frankly). Once 2.x reaches beta, we should probably explore rolling
> new alpha releases to ensure it doesn't rot.
>
> On Sun, May 12, 2013 at 9:26 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
> > You keep twisting around the purpose of the vote and my position in it.
> > As I said before: http://s.apache.org/WBf
> > I am not against the features. And I am not blocking them.
> > I propose to release them in a different than yours order, addressing
> > features vs. stability tradeoff.
>
> It's possible we're confused. Your proposal sounds like Arun should RM
> a release with a particular profile. I apologize if I assumed more
> than you intended.
>
> > I don't know how to stop votes even if I wanted to.
> > As Apache members you should have more vote-stopping power than me if
you
> > think it is against ASF norms.
>
> The content of releases isn't a power struggle. You have all the tools
> and authority you need to create a release. Nobody has authority to
> block you, and frankly, nobody is trying. However, your technical
> input on the particular features would be most welcome. -C

Re: [VOTE] Release plan for Hadoop 2.0.5

[Robert Evans <ev...@yahoo-inc.com>]

Chris,

I realize that my metaphor was not perfect none are.

I don't belive that .snapshots or any other features on the table are bad,
not well tested, or have design flaws. I am just stating that any time
thousands of lines of code are changed for any reason there will be bugs
no matter who wrote it and no matter who tested it.  We are all human and
we will all miss something. I simply want to reduce the time frame between
when code is written and when it is rolled out to be more fully tested. I
see API stabilization as a critical piece of this, and would like to help
the community get in a mindset where we are ready for rolling upgrades
when they do come. If the community thinks that snapshots or some other
feature should go in in the same time frame as API lockdown I am fine with
that. It just means that for me to go to production with that code will in
all likelihood take longer.

I apologize if "junkyard" or "dumping ground" has offended anyone. It was
not my intention to offend, I just wanted to describe what I have seen
happening where a change does not go into branch-2 for some reason
(possibly because of an incompatibility) and it will go into trunk.
Historically it would then sit with little more than unit tests to find
bugs.  I personally don't see any reason for a patch to go into trunk and
not branch-2 unless it breaks compatibility.  And if it breaks
compatibility why is it going into trunk? Is it really that difficult to
maintain compatibility and add new features at the same time? Then if
trunk and branch-2 are essentially identical why would we want to maintain
two copies of the same thing?  These are the questions that I struggle to
find an answer to, and are part of the reason why I am in favor of
Konstantin's proposal.  Although I do agree with you Chris that we need
more clarification in the roles involved with his release plan.

--Bobby

On 5/13/13 4:46 PM, "Chris Douglas" <cd...@apache.org> wrote:

>Bobby-
>
>As much as I enjoy the Evel Knievel image, your argument is not
>finding traction for lack of a visceral metaphor. It's the lack of
>detail. We're not jumping buses, we're adding features to code, where
>it's possible to be specific. If you're uncomfortable with the design,
>implementation, or test plan of a feature, then share your
>reservations. Either someone can reassure you that your issue is
>covered by existing tests, they can add new tests, or- given enough
>evidence- we can agree that the feature needs more time to bake before
>being added to the beta. If you need extra time to do this, please
>insist. Given all that's been written, I literally can't believe that
>nobody has time to do this, and it would be a lot more productive.
>
>I share your concerns about trunk, abstractly. Currently, there's
>almost nothing there that isn't in branch-2 (which makes metaphors
>like "junkyard" and "dumping ground" sound a little hysterical,
>frankly). Once 2.x reaches beta, we should probably explore rolling
>new alpha releases to ensure it doesn't rot.
>
>On Sun, May 12, 2013 at 9:26 PM, Konstantin Shvachko
><sh...@gmail.com> wrote:
>> You keep twisting around the purpose of the vote and my position in it.
>> As I said before: http://s.apache.org/WBf
>> I am not against the features. And I am not blocking them.
>> I propose to release them in a different than yours order, addressing
>> features vs. stability tradeoff.
>
>It's possible we're confused. Your proposal sounds like Arun should RM
>a release with a particular profile. I apologize if I assumed more
>than you intended.
>
>> I don't know how to stop votes even if I wanted to.
>> As Apache members you should have more vote-stopping power than me if
>>you
>> think it is against ASF norms.
>
>The content of releases isn't a power struggle. You have all the tools
>and authority you need to create a release. Nobody has authority to
>block you, and frankly, nobody is trying. However, your technical
>input on the particular features would be most welcome. -C

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Douglas <cd...@apache.org>]

Bobby-

As much as I enjoy the Evel Knievel image, your argument is not
finding traction for lack of a visceral metaphor. It's the lack of
detail. We're not jumping buses, we're adding features to code, where
it's possible to be specific. If you're uncomfortable with the design,
implementation, or test plan of a feature, then share your
reservations. Either someone can reassure you that your issue is
covered by existing tests, they can add new tests, or- given enough
evidence- we can agree that the feature needs more time to bake before
being added to the beta. If you need extra time to do this, please
insist. Given all that's been written, I literally can't believe that
nobody has time to do this, and it would be a lot more productive.

I share your concerns about trunk, abstractly. Currently, there's
almost nothing there that isn't in branch-2 (which makes metaphors
like "junkyard" and "dumping ground" sound a little hysterical,
frankly). Once 2.x reaches beta, we should probably explore rolling
new alpha releases to ensure it doesn't rot.

On Sun, May 12, 2013 at 9:26 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> You keep twisting around the purpose of the vote and my position in it.
> As I said before: http://s.apache.org/WBf
> I am not against the features. And I am not blocking them.
> I propose to release them in a different than yours order, addressing
> features vs. stability tradeoff.

It's possible we're confused. Your proposal sounds like Arun should RM
a release with a particular profile. I apologize if I assumed more
than you intended.

> I don't know how to stop votes even if I wanted to.
> As Apache members you should have more vote-stopping power than me if you
> think it is against ASF norms.

The content of releases isn't a power struggle. You have all the tools
and authority you need to create a release. Nobody has authority to
block you, and frankly, nobody is trying. However, your technical
input on the particular features would be most welcome. -C

Re: [VOTE] Release plan for Hadoop 2.0.5

[Stack <st...@duboce.net>]

On Fri, May 10, 2013 at 11:55 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

>
> On May 9, 2013, at 10:03 PM, Uma Maheswara Rao G wrote:
>
> >  Adding new features really a great thing and surely each big feature
> can be included in one one major release as well.
>
>
> Surely the one thing we have learnt over the last 4-5 years in this
> project is that we cannot make too many major releases. Notice how long
> hadoop-0.20/hadoop-1 lives; the same will happen with hadoop-2.x. Look at
> Bobby's message on this thread on *-dev lists:
>
> >>> Up to this point we have almost successfully done this switch once,
> from 1.0 to 2.0. I have a hard time believing that we are going to do this
> again for another 5 years.
>
> Each major release is a *lot* of work and is, subsequently, an opportunity
> where incompatibilities at various levels creep in - our users are not well
> served by this.
>
> So, it's a  fallacy that we can and should make major releases per feature.
>

To me, the above argument is built on its own fallacies:

+ IMO, hadoop1 is long-lived because there is not yet an hadoop2.
+ My observation is that we as a community have difficulty putting out
major releases.  I do not see that we should NOT be making more frequent
major releases   Smaller scoped major releases should be less work with
less compatibilities, etc.

Good on you Arun,
St.Ack

Re: [VOTE] Release plan for Hadoop 2.0.5

[Robert Evans <ev...@yahoo-inc.com>]

If I have to get to the other end of 14 busses I can either do it Evel
Knievel style and jump them all at once or I can walk from one end to the
other, one step at a time.  I personally would rather take many different
small steps than try to take one giant flying leap and risk missing.

Also I don't think anyone said that we are going to to a major release per
feature.  I thought the proposal was to do a 2.0 release before snapshots
is merged in and a have snapshots go into 2.1.  I don't really want to
open up the version numbering discussion again but I want my quote to be
interpreted the way I intended it to be.  I don't see snapshots as a major
change.  I see it as a step.  I just don't see a reason to keep two
parallel lines branch-2 where we try hard to maintain compatibility, and
trunk where is is a dumping ground that only makes it more difficult to
move to in the future.

--Bobby

On 5/10/13 1:55 PM, "Arun C Murthy" <ac...@hortonworks.com> wrote:

>
>On May 9, 2013, at 10:03 PM, Uma Maheswara Rao G wrote:
>
>>  Adding new features really a great thing and surely each big feature
>>can be included in one one major release as well.
>
>
>Surely the one thing we have learnt over the last 4-5 years in this
>project is that we cannot make too many major releases. Notice how long
>hadoop-0.20/hadoop-1 lives; the same will happen with hadoop-2.x. Look at
>Bobby's message on this thread on *-dev lists:
>
>>>> Up to this point we have almost successfully done this switch once,
>>>>from 1.0 to 2.0. I have a hard time believing that we are going to do
>>>>this again for another 5 years.
>
>Each major release is a *lot* of work and is, subsequently, an
>opportunity where incompatibilities at various levels creep in - our
>users are not well served by this.
>
>So, it's a  fallacy that we can and should make major releases per
>feature.
>
>Arun
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

On May 9, 2013, at 10:03 PM, Uma Maheswara Rao G wrote:

>  Adding new features really a great thing and surely each big feature can be included in one one major release as well.


Surely the one thing we have learnt over the last 4-5 years in this project is that we cannot make too many major releases. Notice how long hadoop-0.20/hadoop-1 lives; the same will happen with hadoop-2.x. Look at Bobby's message on this thread on *-dev lists:

>>> Up to this point we have almost successfully done this switch once, from 1.0 to 2.0. I have a hard time believing that we are going to do this again for another 5 years.

Each major release is a *lot* of work and is, subsequently, an opportunity where incompatibilities at various levels creep in - our users are not well served by this.

So, it's a  fallacy that we can and should make major releases per feature.

Arun

RE: [VOTE] Release plan for Hadoop 2.0.5

[Uma Maheswara Rao G <ma...@huawei.com>]

+1
Sounds like a good plan. Having a release with lot of new features with lot of core code changes will really make users afraid to migrate. Initially I was under the impression that branch-2 will be stabilized soon. But the current strategy looks like, it may take some more time. So, having a cut at this stage and focusing on stabilization will be a good idea.  Adding new features really a great thing and surely each big feature can be included in one one major release as well.

Regards,
Uma  

________________________________________
From: Konstantin Shvachko [shv.hadoop@gmail.com]
Sent: Thursday, May 02, 2013 1:23 AM
To: general@hadoop.apache.org
Subject: [VOTE] Release plan for Hadoop 2.0.5

Please vote on the following plan for Hadoop release 2.0.5
- bug fixes encountered in current release 2.0.4-alpha
- make all API changes to allow freezing them post 2.0.5
- no new features

As discussed on @dev thread
http://s.apache.org/fs
this will allow to stabilize 2.0 branch in a short and predictable period
of time.
This enables a powerful option to have the release tested at Yahoo scale.
The plan is to follow up with 2.1.0 - the stable release.
New features can and should be added on top of the stable release once it
is out.

Hadoop by-laws:
http://hadoop.apache.org/bylaws.html

"Release Plan
Defines the timetable and actions for a release. The plan also nominates a
Release Manager.
Lazy majority of active committers"

assume nomination of a Release Manager with the plan.
It would be really good if Arun continues if this plan is adopted.
We can return to the RM topic if not.

The vote will run for 7 days until next Wed, May 8th.

Thanks,
--Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Steve Loughran <st...@gmail.com>]

On 4 May 2013 18:38, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sat, May 4, 2013 at 3:33 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
>> The proposal sounds like an ideal solution but it is impractical.
>> I think it is hard to make all API changes now and freezing them.
>> Either it will just take a long time to finish the API changes, or
>> we may miss some important API changes.
>
> In fact this was entire point of my comment wrt. high degree of
> focus towards downstream components of anything that could
> be potentially called a Hadoop beta release.
>
> The reality of the situation that we can't simply wish away is
> that Hadoop is not Java -- it doesn't have a formal testsuite
> along the lines of the TCK that can guarantee API stability.

I'm working on filesystem stuff, but that's only tested at the unit
test level, "does it match the requirements", which is different from
"does it work with Pig".

> We don't have that. Hence we might as well use the next
> best thing -- tons of code implemented downstream that
> actually exercise Hadoop APIs.

I'm in favour of this, and do want to tease out some of the swift-only
scale tests for a bigtop patch that we can apply to all filesystems
-they are good at finding bugs. For example, if you create a few
thousand files in a blobstore then try to delete them, that's when you
discover the RESTy endpoints throttle the operations and your code
starts failing unless you insert self-throttling and extend socket
timeouts.  Those are the fun things you want to find sooner rather
than later.

>
> Perhaps a shift of perspective is needed on the part of
> Hadoop community -- we should stop looking at downstream
> as just downstream and start looking at it as a de-facto
> TCK. If we assume that vantage point then things like
> making sure that there are regular Unit tests runs clearly
> become something that useful to Hadoop directly, not
> a 'downstream problem'.

This is something we need to start working on, with Jenkins picking up
problems sooner rather than later.  for it to work -and I say this
based on experience in multi-team projects- everyone needs to care
about problems downstream.

 What I don't want to do -and am -voting against, is Kos's proposal:

1 (non-binding)

There are some things in there -windows support- that shouldn't be
reverted as Kos wants, and I don't want to go near the Hadoop
numbering scheme again. 2.x is 2.x and let's not change our mind now.

As Nicholas says, we need to get some of the protocols and APIs stable
so they can stay forwards compatible. We've also had problems with
HDFS changes in the 2.0 alphas; those wire things need to be locked
down as well as -some- of the APIs. I'm (mostly) happy with management
layer stuff changing, but not the code that users and downstream apps
use.

That doesn't mean 2.x needs to be feature complete, only that everyone
in the core hadoop team is happy that it's got the hooks in there for
the roadmap, and make sure that HDFS v2 is stable and trusted. Because
if you look at the rate of change of Linux, you can say "they
alternate stable/unstable every six months", but the development of
ext4 is much more cautious -it has the same requirement as HDFS:
preserve data you care about.


We should get it out the door, then start to get that bigtop stack
doing the nightly build/test, with failures being reported back to
hadoop-common as well as bigtop. As a reference point, Apache gump was
the first "build everything from SCM" Java project in the ASF,
checking out everything, bootstrapping Ant off its build.sh script and
the JDK, then going into the XML parsers and logging, before trying to
do the rest of the DAG of Java projects. Its goal was portrayed as
"nightly build and tests for your project", but the Ant team viewed it
as our regression tests. Sadly Maven killed it by making it
near-impossible to build an up to date dependency graph.

If we have HDFS stable then YARN and MR can roll at a faster rate,
along with the other downstream parts.

One other thing I want to pick up on is that testing "at Y! scale"
isn't sufficient to get all bugs ironed out -if you look at what
surfaces outside its the people whose networks are a badly configured
mess (mine included), virtualised with very, very odd networks, clock
drift and RAM paging out under the OS's radar, or very different uses.

Those are the problems that will show up in the -beta phase, precisely
because -alpha isn't viewed as safe/stable. I expect some fun bugs to
surface there -bugs we aren't going to see in bigtop either, because
the kind of person who is trying to set up a cluster using the IP
address pool 127.0.1.* isn't the kind of person who is going to check
out bigtop and run nightly tests.

-Steve

Re: [VOTE] Release plan for Hadoop 2.0.5

[Roman Shaposhnik <rv...@apache.org>]

On Sat, May 4, 2013 at 3:33 PM, Tsz Wo Sze <sz...@yahoo.com> wrote:
> The proposal sounds like an ideal solution but it is impractical.
> I think it is hard to make all API changes now and freezing them.
> Either it will just take a long time to finish the API changes, or
> we may miss some important API changes.

In fact this was entire point of my comment wrt. high degree of
focus towards downstream components of anything that could
be potentially called a Hadoop beta release.

The reality of the situation that we can't simply wish away is
that Hadoop is not Java -- it doesn't have a formal testsuite
along the lines of the TCK that can guarantee API stability.

We don't have that. Hence we might as well use the next
best thing -- tons of code implemented downstream that
actually exercise Hadoop APIs.

Perhaps a shift of perspective is needed on the part of
Hadoop community -- we should stop looking at downstream
as just downstream and start looking at it as a de-facto
TCK. If we assume that vantage point then things like
making sure that there are regular Unit tests runs clearly
become something that useful to Hadoop directly, not
a 'downstream problem'.

Thanks,
Roman.

Re: [VOTE] Release plan for Hadoop 2.0.5

[Tsz Wo Sze <sz...@yahoo.com>]

The proposal sounds like an ideal solution but it is impractical.  I think it is hard to make all API changes now and freezing them.  Either it will just take a long time to finish the API changes, or we may miss some important API changes.


Tsz-Wo




________________________________
 From: Konstantin Shvachko <sh...@gmail.com>
To: "general@hadoop.apache.org" <ge...@hadoop.apache.org> 
Sent: Wednesday, May 1, 2013 12:53 PM
Subject: [VOTE] Release plan for Hadoop 2.0.5
 

Please vote on the following plan for Hadoop release 2.0.5
- bug fixes encountered in current release 2.0.4-alpha
- make all API changes to allow freezing them post 2.0.5
- no new features

As discussed on @dev thread
http://s.apache.org/fs
this will allow to stabilize 2.0 branch in a short and predictable period
of time.
This enables a powerful option to have the release tested at Yahoo scale.
The plan is to follow up with 2.1.0 - the stable release.
New features can and should be added on top of the stable release once it
is out.

Hadoop by-laws:
http://hadoop.apache.org/bylaws.html

"Release Plan
Defines the timetable and actions for a release. The plan also nominates a
Release Manager.
Lazy majority of active committers"

assume nomination of a Release Manager with the plan.
It would be really good if Arun continues if this plan is adopted.
We can return to the RM topic if not.

The vote will run for 7 days until next Wed, May 8th.

Thanks,
--Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Stack <st...@duboce.net>]

On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
<sh...@gmail.com>wrote:

> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
>

+1

I love the new features but can wait.

St.Ack

RE: [VOTE] Release plan for Hadoop 2.0.5

[Jagane Sundar <ja...@sundar.org>]

+1 (non-binding) for adopting the 'fix bugs and include all changes required for freezing APIs, but defer invasive feature changes to a follow on release'

(I would like to preface my comments by stating that I have a great deal of respect and admiration for the folks who created and developed Hadoop to where it is today, and it is definitely not my intent to declare how Apache Hadoop releases should be made, or to proclaim mastery of producing a stable Hadoop release)

There is no question that it would be of great value to the Hadoop community to have a stable, standard Apache Hadoop 2.0 based core. The proposal that Konst is making would take a specific path to this goal. I agree with Konst in that this is probably the shortest and least risky path to a stable 2.0.

The advantages as I see it are:
1. Large deployments such as Y! that were instrumental in the amazing product that is Hadoop may feel comfortable moving to Hadoop 2.0 in calendar 2013 if we take Konst's low risk approach
2. Downstream projects such as HBase will benefit greatly from an API stable 2.0 - the sooner the better.
3. A stable Hadoop 2.0 with all of its new features will enable Hadoop to maintain its lead against other competing open source projects such as Cassandra

The disadvantage in deferring HDFS Snapshots, HDFS NFS gateway and Hadoop Windows support to a follow on release is obvious. I have the utmost respect for the engineers who are actively developing these complex features - however, it takes time for software to mature.

Thanks,
Jagane

-----Original Message-----
From: Konstantin Shvachko [mailto:shv.hadoop@gmail.com] 
Sent: Wednesday, May 01, 2013 12:54 PM
To: general@hadoop.apache.org
Subject: [VOTE] Release plan for Hadoop 2.0.5

Please vote on the following plan for Hadoop release 2.0.5
- bug fixes encountered in current release 2.0.4-alpha
- make all API changes to allow freezing them post 2.0.5
- no new features

As discussed on @dev thread
http://s.apache.org/fs
this will allow to stabilize 2.0 branch in a short and predictable period of time.
This enables a powerful option to have the release tested at Yahoo scale.
The plan is to follow up with 2.1.0 - the stable release.
New features can and should be added on top of the stable release once it is out.

Hadoop by-laws:
http://hadoop.apache.org/bylaws.html

"Release Plan
Defines the timetable and actions for a release. The plan also nominates a Release Manager.
Lazy majority of active committers"

assume nomination of a Release Manager with the plan.
It would be really good if Arun continues if this plan is adopted.
We can return to the RM topic if not.

The vote will run for 7 days until next Wed, May 8th.

Thanks,
--Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Steve Loughran <st...@gmail.com>]

On 8 May 2013 22:40, Konstantin Shvachko <sh...@gmail.com> wrote:

> My formal +1.
>
> Vote passes with
> 3 binding +1s
> 1 non-binding +1
> 1 binding -1
> 1 non-binding -1
> and one (Steve's) vote which I couldn't categorize: 1 (non-binding)
>
>
typo:. -1 non binding

I do support having bigtop in the release process, but not this
over-aggressive triage of features that seems more based on likes/dislikes
than sources of trouble.

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Oh. And it would be good to have an explanation for -1 votes.
Only one negative vote came with a reason so far.

Thanks,
--Konstantin


On Thu, May 9, 2013 at 7:58 AM, Suresh Srinivas <su...@hortonworks.com>wrote:

> I agree it was premature to start this vote and I also thought it was going
> to be restarted given
> the discussions Nicholas, Roman and Steve had started on this thread.
>
> I am -1 (binding) on this.
>
>
> On Thu, May 9, 2013 at 6:12 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> > Konstantin,
> >
> >  You know I've been strongly opposed to this due to the bad precedents it
> > sets; as I've made clear publicly in the *-dev lists.
> >
> >  I was under impression you would restart this vote, but my bad.
> >
> >  I am going to -1 this (binding).
> >
> >  Could you please consider my vote or should I start a new one to unroll
> > this?
> >
> > thanks,
> > Arun
> >
> > On May 8, 2013, at 10:40 PM, Konstantin Shvachko wrote:
> >
> > > My formal +1.
> > >
> > > Vote passes with
> > > 3 binding +1s
> > > 1 non-binding +1
> > > 1 binding -1
> > > 1 non-binding -1
> > > and one (Steve's) vote which I couldn't categorize: 1 (non-binding)
> > >
> > > Thanks,
> > > --Konstantin
> > >
> > >
> > >
> > > On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
> > > <sh...@gmail.com>wrote:
> > >
> > >> Please vote on the following plan for Hadoop release 2.0.5
> > >> - bug fixes encountered in current release 2.0.4-alpha
> > >> - make all API changes to allow freezing them post 2.0.5
> > >> - no new features
> > >>
> > >> As discussed on @dev thread
> > >> http://s.apache.org/fs
> > >> this will allow to stabilize 2.0 branch in a short and predictable
> > period
> > >> of time.
> > >> This enables a powerful option to have the release tested at Yahoo
> > scale.
> > >> The plan is to follow up with 2.1.0 - the stable release.
> > >> New features can and should be added on top of the stable release once
> > it
> > >> is out.
> > >>
> > >> Hadoop by-laws:
> > >> http://hadoop.apache.org/bylaws.html
> > >>
> > >> "Release Plan
> > >> Defines the timetable and actions for a release. The plan also
> > nominates a
> > >> Release Manager.
> > >> Lazy majority of active committers"
> > >>
> > >> assume nomination of a Release Manager with the plan.
> > >> It would be really good if Arun continues if this plan is adopted.
> > >> We can return to the RM topic if not.
> > >>
> > >> The vote will run for 7 days until next Wed, May 8th.
> > >>
> > >> Thanks,
> > >> --Konstantin
> > >>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
>
>
> --
> http://hortonworks.com/download/
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Milind Bhandarkar <mb...@gopivotal.com>]

+1 (non-binding)
Would love to see feature-freeze soon in 2.0.x, and so folks can declare it
stable a day before hadoop summit, 2013.

- Milind

On Thu, May 9, 2013 at 2:41 PM, Arun C Murthy <ac...@hortonworks.com> wrote:

>
> On May 9, 2013, at 2:31 PM, Konstantin Shvachko wrote:
> >
> >
> > If you need more time sure let's let everybody vote.
> > The vote will run until Monday May 13.
>
> Sounds reasonable. Thanks Konst.
>
> Arun
>
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

On May 9, 2013, at 2:31 PM, Konstantin Shvachko wrote:
> 
> 
> If you need more time sure let's let everybody vote.
> The vote will run until Monday May 13. 

Sounds reasonable. Thanks Konst.

Arun

Re: [VOTE] Release plan for Hadoop 2.0.5

[Andrew Purtell <ap...@apache.org>]

It would seem to a humble outsider that project formalism and procedure is
not the issue, instead that is expectations and impact on the outside
world. We hear that branch-2 is approaching stability, except when it
isn't, as evidenced by new downstream project unit test failures at each
"minor" release, with major new features going in because it's too
difficult to renumber. (??)

Would it work if a consensus can be found on renumbering upcoming 2.0.5 to
2.1.0? That would be a simple and effective concrete step toward resolving
the disagreements I observe. Those who have voted +1 on this
discussion+vote thread can stabilize 2.0.4, with additional fixes, RM a
2.0.x branch, describe it as stable, and meanwhile life goes on for
everyone else on 2.1.x branch? Downstream projects could be advised to
stick with 2.0.x for API and runtime stability until ready to make the leap
up to 2.1.x or successor?

On Sat, May 11, 2013 at 4:34 AM, Chris Douglas <cd...@apache.org> wrote:

> It's unnecessary for you to ask permission to roll a release
> containing (or omitting) the features you want. This vote is redundant
> with the release vote; it's an unnecessary formalism in our bylaws. If
> you want to release 2.0.5 with the features you want and assemble
> other community members to help stabilize it in a 2.0.x series...
> great. Do that. If the release vote passes and others want to add more
> features, then they can roll a new series in 2.1. If the vote on your
> artifact fails, then forward your "stability" agenda directly by
> reviewing and contributing code. If a set of contributors wants to
> support your agenda then they'll work on it, but if the developers
> aren't there then neither is the project.
>




-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Hi Chris and Arun,

You keep twisting around the purpose of the vote and my position in it.
As I said before: http://s.apache.org/WBf
I am not against the features. And I am not blocking them.
I propose to release them in a different than yours order, addressing
features vs. stability tradeoff.

Arun, you should remember that I started this argument with the proposal to
shift versions.
Just as Andrew proposed in his last email.
But you rejected the idea at the time.
This would have let people who want to work on stable 2.0 series, while you
could have continued releasing 2.1 with more features.

I think the lesson Arun learned from the longevity of branch-1 is wrong.
The lesson I learn is that if you keep one branch for too long you get
others branch off and you start loosing important members of the community
like Facebook. I think we should produce feature releases from trunk and
often. With the stability of a particular branch coming later when there is
a critical mass of contributors to work on its stabilization.

I don't know how to stop votes even if I wanted to.
As Apache members you should have more vote-stopping power than me if you
think it is against ASF norms.

Thanks,
--Konstantin


On Sat, May 11, 2013 at 7:03 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

> Chris,
>
>  I couldn't agree more with the sentiments - thank you for expressing them
> in such a lucid manner.
>
>  There is one nit I'd like to point out though:
>
> On May 10, 2013, at 1:34 PM, Chris Douglas wrote:
>
> On Thu, May 9, 2013 at 11:14 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
>
> Not everybody who is voting now provided context in the discussion thread.
>
> You did. And I am sorry I did not understand it.
>
>
> I'll try to be clearer.
>
> It's unnecessary for you to ask permission to roll a release
> containing (or omitting) the features you want. This vote is redundant
> with the release vote; it's an unnecessary formalism in our bylaws. If
> you want to release 2.0.5 with the features you want and assemble
> other community members to help stabilize it in a 2.0.x series...
> great. Do that.
>
>
> This is the nit.
>
> In the ASF, the RM *does not* have the power to choose bits and pieces of
> code from SVN. He can remove bits from SVN - only by veto'ing the changes.
>
> Roy was very clear on this on this very list: http://s.apache.org/Gt9
>
> I'll quote directly from him -
>
> The only thing the RM has authority over is the building of a source
> package, based on the contents of our subversion, that can then be
> put up for vote.  They can decide what snapshot to tag for a build.
> They can decide not to build anything at all.  They can also do all sorts
> of organizational support, advocacy, pleading, or whatever in order to
> encourage the rest of the project committers to apply changes, vote
> for things under issue, etc.
>
> They do not have the right to pick and select whatever variation
> of the product they might like to build, short of vetoing (with a
> valid reason) any changes that they as a PMC member believe do not
> belong on the branch. Likewise, the RM cannot include in the build
> any change that has been vetoed by others, and their build cannot
> be released if it contains any such changes that have been vetoed
> since it was built.  The RM has the right to kill their own build
> if they learn something during the release process that they think,
> for whatever reason, causes the build to be unreleasable.
>
>
> Furthermore, this vote is, essentially, against the rules on the ASF since
> it's trying to block changes into a specific branch (i.e. branch-2).
>
> Again, quoting Roy from the same email:
>
> Any committer can commit wherever they have been given permission
> to commit by the PMC.  Generally, they do so collaboratively.
> I've never encountered a situation in my own projects which developers
> were committing at cross-purposes, even when they disagree on content,
> though I've seen commit wars elsewhere.  We'd expect the PMC
> to step in if they did.
>
>
> ----
>
> In all, Konstantin - can you please stop this vote?
>
> As I have repeatedly pleaded with you - please work with developers
> working on the issues you seem set against (HDFS-347, Snapshots, Windows
> support) and come to a *consensus*. That is the only way to reach the goals
> you desire.
>
> You are welcome to veto/revert any of the changes from all of Apache
> Hadoop subversion with a valid technical reason.
>
>  Again, we don't- we can't- assign work by voting. This is a poll, and a
> feckless one.
>
>
> I'll repeat, this vote doesn't make any sense (regardless of the artistic
> words in our bylaws which need to change/go). This is more of a populistic
> poll which tries to sway people with horror stories of instability,
> non-convergence of hadoop-2 to a stable state etc. etc.
>
> IAC, this vote is against the norms of the ASF, please end this.
>
> thanks,
> Arun
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

On May 13, 2013, at 8:33 AM, Chris Douglas wrote:

> On Sat, May 11, 2013 at 7:03 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> In the ASF, the RM *does not* have the power to choose bits and pieces of code from SVN. He can remove bits from SVN - only by veto'ing the changes. [ample quotation of Roy]
> 
> A committer can create a branch, push changes to it, and invite others
> to work on it. He may subsequently propose the contents of that branch
> as a release, begging, convincing, etc. others in the context of a
> "release manager". Whether the branching, coding, and testing is done
> in an "RM context" doesn't seem particularly important to its
> feasibility. But your point is well taken, and it's important to
> identify this path as developers forking the branch, not an RM
> curating a release. And few would disagree: it's better for developers
> to collaborate, rather than working at cross purposes in related
> branches.
> 

Glad we are on the same page.


> However, the course Andrew (and others) have advocated, where recently
> committed features go into a 2.1.x release series instead of 2.0.x, is
> not disallowed by any rule.

First up - agree 110%

We seem to have an outstanding ability to replay discussions every couple of months here. Pardon me as my frustration bubbles over.

I originally proposed something along the same lines in Feb 2012 i.e. 2.1.x as alpha series, 2.2.x as beta series etc.
http://s.apache.org/tj3 

In the long, windy discussion that (invariably) ensued it was shot down in favor of the current plan - I wasn't thrilled, but I just moved on:
http://s.apache.org/BmM

I pointed this out in the other thread on *-dev@ too; this keeps getting ignored as we pass each other on the concentric circles we currently route on.

Again, I don't care whether beta is 2.1.x or 2.2.x or 2.100.x. All I care is to have a set of well-known series of releases (history shows we'll need at least two or more) to whet APIs and compatibility *before* we declare them to be done.

So, yes, I'm more than happy to re-number 2.0.5 to 2.1 or 2.100 and abandon the 2.0.x series.

However, I want to make sure it's clear to everyone that by doing so, there will be incompatibilities between 2.0.x and 2.1.x; hence 2.0.x will continue along a path where it won't be feasible to support in a stable/compatible manner for the long term i.e. continue to be called 'alpha' in terms of API/protocol compatibility - that's it. 

As others (Suresh, Nic, Eli) have pointed out, declaring APIs/protocols done while we are days away from merging major features such as Snapshots is a fool's errand - we are better off taking them in. That is even more so given the importance of the feature and the number of who people stand by it, please look at the test plans (if anyone disagrees comment on the jira, not on this thread).

Todd claims HDFS-347 is stable (see recent discussion on jira), I believe him. 

I know the Windows stuff is low risk - I've looked at the changes to the core of MR/YARN; they are isolated and well outside risk perimeters which concern me. Among the biggest changes was to add WindowsContainerExecutor for crying out loud - to those unfamiliar, this plugin will *not* be used in other platforms. So, yes, call me skeptical when people fret over this. I believe HDFS changes are similarly low-risk and have been already verified on branch-1/trunk - again, please, comment on jira if it concerns you.

Arun

Re: [VOTE] Release plan for Hadoop 2.0.5

[Andrew Purtell <ap...@apache.org>]

I don't understand why this should become about whether a downstream
project follows upstream JIRA arcania. In fact some of us do, and even
participate occasionally in discussions here. The HBase community is
particularly active in this regard. Perhaps a brief survey of JIRA history
will confirm that to your satisfaction. I also find baseless the ad hominem
that somehow I am indulging in fantasy or .. I'm not exactly sure what. I
came here to respectfully perform exactly the participation you implore
from a downstream project point of view. It would be great if the further
evolution of 2.0.x could continue to stabilize without further major
internal changes. This does not ask that Hadoop stop developing features,
or to stop backporting to branch-2 (on a 2.1.x or ...)

On Monday, May 13, 2013, Chris Douglas wrote:

>
> On Fri, May 10, 2013 at 5:48 PM, Andrew Purtell <apurtell@apache.org<https://mail.google.com/mail/mu/mp/2/?source=nap&hr=1&hl=en>>
> wrote
> > It would seem to a humble outsider that project formalism and procedure
> is
> > not the issue, instead that is expectations and impact on the outside
> > world. We hear that branch-2 is approaching stability, except when it
> > isn't, as evidenced by new downstream project unit test failures at each
> > "minor" release, with major new features going in because it's too
> > difficult to renumber. (??)
>
> Respectfully, this is against software tagged so explicitly as
> "alpha", it's in the version number. Did you file JIRAs and
> participate in reviewing the protocols/APIs that changed? Did you
> raise your use cases with developers, so they know someone was coding
> against that behavior?
>
> The 2.x branch contains years of work from its contributors, who know
> how fortunate they are to have downstream projects eager to use them.
> But let's not indulge in the fantasy that all our frustrations with
> software development are due to others' negligence. If you follow JIRA
> traffic, the work on ensuring that branch-2 protocols remain stable
> has been the source of most compatibility issues between patch
> versions. And several recent discussions have covered how Bigtop can
> help Hadoop development know when it breaks downstream projects. -C
>


-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Douglas <cd...@apache.org>]

On Sat, May 11, 2013 at 7:03 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> In the ASF, the RM *does not* have the power to choose bits and pieces of code from SVN. He can remove bits from SVN - only by veto'ing the changes. [ample quotation of Roy]

A committer can create a branch, push changes to it, and invite others
to work on it. He may subsequently propose the contents of that branch
as a release, begging, convincing, etc. others in the context of a
"release manager". Whether the branching, coding, and testing is done
in an "RM context" doesn't seem particularly important to its
feasibility. But your point is well taken, and it's important to
identify this path as developers forking the branch, not an RM
curating a release. And few would disagree: it's better for developers
to collaborate, rather than working at cross purposes in related
branches.

However, the course Andrew (and others) have advocated, where recently
committed features go into a 2.1.x release series instead of 2.0.x, is
not disallowed by any rule. Whenever we've tried to assert that ASF
rules "require" us to accept or reject code, it pushes development
outside of Apache. What needs to happen at this stage is a
negotiation, not a wizards' duel on the bylaws. As Bobby pointed out
in the other thread, stabilization is an active process not a passive
one. We'll all be more successful if we can (a) tranquilize anxiety
about each new feature (b) encourage feature authors to participate in
that.

----

Summarizing considerably, Konstantin (and others) are anxious to see
2.x stabilize and to ensure trunk stays releasable. One strategy he
proposed: stage large, complex features in more frequent releases. To
come up with alternative proposals that achieve these aims, others
need more context. For all that's been written about software
development generally, there are few details on what, exactly, is
harming stability.

So: for a given feature, are there holes in the test plan? Do you have
benchmarks you'd like to run? What difficult edge cases might have
been missed? If you're not comfortable with the design or
implementation, what action would reassure you that it's ready for a
beta release? If you're coding a downstream project, are the unit
tests covering behaviors you rely on? Are the JIRAs defining
compatibility covering your use of those APIs?

If you're worried about stability of new features: it's legal to fork,
but there are many reasons why that time is better spent helping
others to harden the beta release.

On Fri, May 10, 2013 at 5:48 PM, Andrew Purtell <ap...@apache.org> wrote:
> It would seem to a humble outsider that project formalism and procedure is
> not the issue, instead that is expectations and impact on the outside
> world. We hear that branch-2 is approaching stability, except when it
> isn't, as evidenced by new downstream project unit test failures at each
> "minor" release, with major new features going in because it's too
> difficult to renumber. (??)

Respectfully, this is against software tagged so explicitly as
"alpha", it's in the version number. Did you file JIRAs and
participate in reviewing the protocols/APIs that changed? Did you
raise your use cases with developers, so they know someone was coding
against that behavior?

The 2.x branch contains years of work from its contributors, who know
how fortunate they are to have downstream projects eager to use them.
But let's not indulge in the fantasy that all our frustrations with
software development are due to others' negligence. If you follow JIRA
traffic, the work on ensuring that branch-2 protocols remain stable
has been the source of most compatibility issues between patch
versions. And several recent discussions have covered how Bigtop can
help Hadoop development know when it breaks downstream projects. -C

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

Chris,

 I couldn't agree more with the sentiments - thank you for expressing them in such a lucid manner.

 There is one nit I'd like to point out though:

On May 10, 2013, at 1:34 PM, Chris Douglas wrote:

> On Thu, May 9, 2013 at 11:14 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
>> Not everybody who is voting now provided context in the discussion thread.
>> You did. And I am sorry I did not understand it.
> 
> I'll try to be clearer.
> 
> It's unnecessary for you to ask permission to roll a release
> containing (or omitting) the features you want. This vote is redundant
> with the release vote; it's an unnecessary formalism in our bylaws. If
> you want to release 2.0.5 with the features you want and assemble
> other community members to help stabilize it in a 2.0.x series...
> great. Do that.

This is the nit.

In the ASF, the RM *does not* have the power to choose bits and pieces of code from SVN. He can remove bits from SVN - only by veto'ing the changes.

Roy was very clear on this on this very list: http://s.apache.org/Gt9

I'll quote directly from him -

>> The only thing the RM has authority over is the building of a source
>> package, based on the contents of our subversion, that can then be
>> put up for vote.  They can decide what snapshot to tag for a build.
>> They can decide not to build anything at all.  They can also do all sorts
>> of organizational support, advocacy, pleading, or whatever in order to
>> encourage the rest of the project committers to apply changes, vote
>> for things under issue, etc.
>> 
>> They do not have the right to pick and select whatever variation
>> of the product they might like to build, short of vetoing (with a
>> valid reason) any changes that they as a PMC member believe do not
>> belong on the branch. Likewise, the RM cannot include in the build
>> any change that has been vetoed by others, and their build cannot
>> be released if it contains any such changes that have been vetoed
>> since it was built.  The RM has the right to kill their own build
>> if they learn something during the release process that they think,
>> for whatever reason, causes the build to be unreleasable.  

Furthermore, this vote is, essentially, against the rules on the ASF since it's trying to block changes into a specific branch (i.e. branch-2).

Again, quoting Roy from the same email:

>> Any committer can commit wherever they have been given permission
>> to commit by the PMC.  Generally, they do so collaboratively.
>> I've never encountered a situation in my own projects which developers
>> were committing at cross-purposes, even when they disagree on content,
>> though I've seen commit wars elsewhere.  We'd expect the PMC
>> to step in if they did.

----

In all, Konstantin - can you please stop this vote? 

As I have repeatedly pleaded with you - please work with developers working on the issues you seem set against (HDFS-347, Snapshots, Windows support) and come to a *consensus*. That is the only way to reach the goals you desire. 

You are welcome to veto/revert any of the changes from all of Apache Hadoop subversion with a valid technical reason.

>  Again, we don't- we can't- assign work by voting. This is a poll, and a feckless one.


I'll repeat, this vote doesn't make any sense (regardless of the artistic words in our bylaws which need to change/go). This is more of a populistic poll which tries to sway people with horror stories of instability, non-convergence of hadoop-2 to a stable state etc. etc.

IAC, this vote is against the norms of the ASF, please end this.

thanks,
Arun

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Douglas <cd...@apache.org>]

On Thu, May 9, 2013 at 11:14 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Not everybody who is voting now provided context in the discussion thread.
> You did. And I am sorry I did not understand it.

I'll try to be clearer.

It's unnecessary for you to ask permission to roll a release
containing (or omitting) the features you want. This vote is redundant
with the release vote; it's an unnecessary formalism in our bylaws. If
you want to release 2.0.5 with the features you want and assemble
other community members to help stabilize it in a 2.0.x series...
great. Do that. If the release vote passes and others want to add more
features, then they can roll a new series in 2.1. If the vote on your
artifact fails, then forward your "stability" agenda directly by
reviewing and contributing code. If a set of contributors wants to
support your agenda then they'll work on it, but if the developers
aren't there then neither is the project.

Again, we don't- we can't- assign work by voting. This is a poll, and
a feckless one.

I voted -1 because I find voting on an agenda that Arun will implement
absurd, particularly when he disagrees with it. The result is
meaningless, but losing this "vote" would at least conclude these
debates.

> Formally, bylaws void only vetos with no explanations.
> So I agree it is not required in this vote as there are no vetos.
> Wouldn't it be good to know though.

Agreed, but this thread is absorbing that discussion because it hadn't
reached consensus, or anything close to it. If consensus proves
impossible and you still feel it's worth pressing the point, then
create a branch, roll a release, and we'll vote on that artifact as
coherent with the direction of the project. You don't need to ask for
anyone's blessing, but you do need to convince others of your
position. With that in mind, you may reconsider whether peers' code
"destructive", "destabilizing" "junk" forwards your agenda most
effectively. -C

> On Thu, May 9, 2013 at 3:08 PM, Chris Douglas <cd...@apache.org> wrote:
>
>> On Thu, May 9, 2013 at 2:31 PM, Konstantin Shvachko
>> <sh...@gmail.com> wrote:
>> > This is the voting thread, not the discussion one.
>> > The discussion was going on the dev thread.
>> [...]
>> > Oh. And it would be good to have an explanation for -1 votes.
>> > Only one negative vote came with a reason so far.
>>
>> As you point out, we have discussion threads because establishing
>> intent in voting threads is ambiguous. Everyone voting in this thread
>> provided context in the other, though no explanation is required. -C
>>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Chris,

Not everybody who is voting now provided context in the discussion thread.
You did. And I am sorry I did not understand it.
Formally, bylaws void only vetos with no explanations.
So I agree it is not required in this vote as there are no vetos.
Wouldn't it be good to know though.

--Konst


On Thu, May 9, 2013 at 3:08 PM, Chris Douglas <cd...@apache.org> wrote:

> On Thu, May 9, 2013 at 2:31 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
> > This is the voting thread, not the discussion one.
> > The discussion was going on the dev thread.
> [...]
> > Oh. And it would be good to have an explanation for -1 votes.
> > Only one negative vote came with a reason so far.
>
> As you point out, we have discussion threads because establishing
> intent in voting threads is ambiguous. Everyone voting in this thread
> provided context in the other, though no explanation is required. -C
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Douglas <cd...@apache.org>]

On Thu, May 9, 2013 at 2:31 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> This is the voting thread, not the discussion one.
> The discussion was going on the dev thread.
[...]
> Oh. And it would be good to have an explanation for -1 votes.
> Only one negative vote came with a reason so far.

As you point out, we have discussion threads because establishing
intent in voting threads is ambiguous. Everyone voting in this thread
provided context in the other, though no explanation is required. -C

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Hello guys,

This is the voting thread, not the discussion one.
The discussion was going on the dev thread.
I thought the discussion was over with the latest two posts here
*http://s.apache.org/yQ*

Saying "premature" seems to be a wrong API to reset the vote.
Not compatible with Bylaws APIs anyways. As last released.
I was just asking to play by the rules we built for ourselves.

If you need more time sure let's let everybody vote.
The vote will run until Monday May 13. Please chime in if even more time is
needed.
I'll count Arun's and Suresh's votes as voted unless you want to change
them.

I really encourage people to vote as this is about the way we stabilize
Hadoop 2.
See the dev thread for details.
The Hadoop community is not just committers, our jiras, and mailing lists.
I keep learning about new products out there relying on Hadoop APIs every
week.

Thanks,
--Konstantin


On Thu, May 9, 2013 at 7:58 AM, Suresh Srinivas <su...@hortonworks.com>wrote:

> I agree it was premature to start this vote and I also thought it was going
> to be restarted given
> the discussions Nicholas, Roman and Steve had started on this thread.
>
> I am -1 (binding) on this.
>
>
> On Thu, May 9, 2013 at 6:12 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> > Konstantin,
> >
> >  You know I've been strongly opposed to this due to the bad precedents it
> > sets; as I've made clear publicly in the *-dev lists.
> >
> >  I was under impression you would restart this vote, but my bad.
> >
> >  I am going to -1 this (binding).
> >
> >  Could you please consider my vote or should I start a new one to unroll
> > this?
> >
> > thanks,
> > Arun
> >
> > On May 8, 2013, at 10:40 PM, Konstantin Shvachko wrote:
> >
> > > My formal +1.
> > >
> > > Vote passes with
> > > 3 binding +1s
> > > 1 non-binding +1
> > > 1 binding -1
> > > 1 non-binding -1
> > > and one (Steve's) vote which I couldn't categorize: 1 (non-binding)
> > >
> > > Thanks,
> > > --Konstantin
> > >
> > >
> > >
> > > On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
> > > <sh...@gmail.com>wrote:
> > >
> > >> Please vote on the following plan for Hadoop release 2.0.5
> > >> - bug fixes encountered in current release 2.0.4-alpha
> > >> - make all API changes to allow freezing them post 2.0.5
> > >> - no new features
> > >>
> > >> As discussed on @dev thread
> > >> http://s.apache.org/fs
> > >> this will allow to stabilize 2.0 branch in a short and predictable
> > period
> > >> of time.
> > >> This enables a powerful option to have the release tested at Yahoo
> > scale.
> > >> The plan is to follow up with 2.1.0 - the stable release.
> > >> New features can and should be added on top of the stable release once
> > it
> > >> is out.
> > >>
> > >> Hadoop by-laws:
> > >> http://hadoop.apache.org/bylaws.html
> > >>
> > >> "Release Plan
> > >> Defines the timetable and actions for a release. The plan also
> > nominates a
> > >> Release Manager.
> > >> Lazy majority of active committers"
> > >>
> > >> assume nomination of a Release Manager with the plan.
> > >> It would be really good if Arun continues if this plan is adopted.
> > >> We can return to the RM topic if not.
> > >>
> > >> The vote will run for 7 days until next Wed, May 8th.
> > >>
> > >> Thanks,
> > >> --Konstantin
> > >>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
>
>
> --
> http://hortonworks.com/download/
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Suresh Srinivas <su...@hortonworks.com>]

I agree it was premature to start this vote and I also thought it was going
to be restarted given
the discussions Nicholas, Roman and Steve had started on this thread.

I am -1 (binding) on this.


On Thu, May 9, 2013 at 6:12 AM, Arun C Murthy <ac...@hortonworks.com> wrote:

> Konstantin,
>
>  You know I've been strongly opposed to this due to the bad precedents it
> sets; as I've made clear publicly in the *-dev lists.
>
>  I was under impression you would restart this vote, but my bad.
>
>  I am going to -1 this (binding).
>
>  Could you please consider my vote or should I start a new one to unroll
> this?
>
> thanks,
> Arun
>
> On May 8, 2013, at 10:40 PM, Konstantin Shvachko wrote:
>
> > My formal +1.
> >
> > Vote passes with
> > 3 binding +1s
> > 1 non-binding +1
> > 1 binding -1
> > 1 non-binding -1
> > and one (Steve's) vote which I couldn't categorize: 1 (non-binding)
> >
> > Thanks,
> > --Konstantin
> >
> >
> >
> > On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
> > <sh...@gmail.com>wrote:
> >
> >> Please vote on the following plan for Hadoop release 2.0.5
> >> - bug fixes encountered in current release 2.0.4-alpha
> >> - make all API changes to allow freezing them post 2.0.5
> >> - no new features
> >>
> >> As discussed on @dev thread
> >> http://s.apache.org/fs
> >> this will allow to stabilize 2.0 branch in a short and predictable
> period
> >> of time.
> >> This enables a powerful option to have the release tested at Yahoo
> scale.
> >> The plan is to follow up with 2.1.0 - the stable release.
> >> New features can and should be added on top of the stable release once
> it
> >> is out.
> >>
> >> Hadoop by-laws:
> >> http://hadoop.apache.org/bylaws.html
> >>
> >> "Release Plan
> >> Defines the timetable and actions for a release. The plan also
> nominates a
> >> Release Manager.
> >> Lazy majority of active committers"
> >>
> >> assume nomination of a Release Manager with the plan.
> >> It would be really good if Arun continues if this plan is adopted.
> >> We can return to the RM topic if not.
> >>
> >> The vote will run for 7 days until next Wed, May 8th.
> >>
> >> Thanks,
> >> --Konstantin
> >>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>


-- 
http://hortonworks.com/download/

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

Konstantin,

 You know I've been strongly opposed to this due to the bad precedents it sets; as I've made clear publicly in the *-dev lists.

 I was under impression you would restart this vote, but my bad.

 I am going to -1 this (binding).

 Could you please consider my vote or should I start a new one to unroll this?

thanks,
Arun

On May 8, 2013, at 10:40 PM, Konstantin Shvachko wrote:

> My formal +1.
> 
> Vote passes with
> 3 binding +1s
> 1 non-binding +1
> 1 binding -1
> 1 non-binding -1
> and one (Steve's) vote which I couldn't categorize: 1 (non-binding)
> 
> Thanks,
> --Konstantin
> 
> 
> 
> On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
> <sh...@gmail.com>wrote:
> 
>> Please vote on the following plan for Hadoop release 2.0.5
>> - bug fixes encountered in current release 2.0.4-alpha
>> - make all API changes to allow freezing them post 2.0.5
>> - no new features
>> 
>> As discussed on @dev thread
>> http://s.apache.org/fs
>> this will allow to stabilize 2.0 branch in a short and predictable period
>> of time.
>> This enables a powerful option to have the release tested at Yahoo scale.
>> The plan is to follow up with 2.1.0 - the stable release.
>> New features can and should be added on top of the stable release once it
>> is out.
>> 
>> Hadoop by-laws:
>> http://hadoop.apache.org/bylaws.html
>> 
>> "Release Plan
>> Defines the timetable and actions for a release. The plan also nominates a
>> Release Manager.
>> Lazy majority of active committers"
>> 
>> assume nomination of a Release Manager with the plan.
>> It would be really good if Arun continues if this plan is adopted.
>> We can return to the RM topic if not.
>> 
>> The vote will run for 7 days until next Wed, May 8th.
>> 
>> Thanks,
>> --Konstantin
>> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

My formal +1.

Vote passes with
3 binding +1s
1 non-binding +1
1 binding -1
1 non-binding -1
and one (Steve's) vote which I couldn't categorize: 1 (non-binding)

Thanks,
--Konstantin



On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
<sh...@gmail.com>wrote:

> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
>
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
>
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
>
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
>
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
>
> The vote will run for 7 days until next Wed, May 8th.
>
> Thanks,
> --Konstantin
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Boudnik <co...@apache.org>]

Yes, I will be still +1 because there's no other way of fully testing
compatibility of Hadoop with downstream projects but using Bigtop.

Cos

On Sat, May 04, 2013 at 03:21PM, Tsz Wo Sze wrote:
> Then, would you still +1 without the addition?═ If yes, you should -1 on this VOTE.
> Tsz-Wo
> 
> ________________________________
>  From: Konstantin Boudnik <co...@apache.org>
> To: general@hadoop.apache.org 
> Sent: Thursday, May 2, 2013 8:40 PM
> Subject: Re: [VOTE] Release plan for Hadoop 2.0.5
>  
> 
> +1 with one addition I'd like to make to the plan
> 
> ═ - making downstream testing a passing criteria for the release. I don't mean
> ═ ═ that Hadoop community needs to be responsible for testing of _all_
> ═ ═ downstream components. What I'd like to see is a practice we used in the
> ═ ═ 2.0.4-alpha with Bigtop 0.6 where passage of the tests in the latter was
> ═ ═ a, perhaps unofficial, release criteria for the former.
> 
> I also agree with 2.0.5 versioning, because essentially, future stability of
> the API - of freezing - is the solid base we all are looking for.
> 
> Cos
> 
> On Wed, May 01, 2013 at 12:53PM, Konstantin Shvachko wrote:
> > Please vote on the following plan for Hadoop release 2.0.5
> > - bug fixes encountered in current release 2.0.4-alpha
> > - make all API changes to allow freezing them post 2.0.5
> > - no new features
> > 
> > As discussed on @dev thread
> > http://s.apache.org/fs
> > this will allow to stabilize 2.0 branch in a short and predictable period
> > of time.
> > This enables a powerful option to have the release tested at Yahoo scale.
> > The plan is to follow up with 2.1.0 - the stable release.
> > New features can and should be added on top of the stable release once it
> > is out.
> > 
> > Hadoop by-laws:
> > http://hadoop.apache.org/bylaws.html
> > 
> > "Release Plan
> > Defines the timetable and actions for a release. The plan also nominates a
> > Release Manager.
> > Lazy majority of active committers"
> > 
> > assume nomination of a Release Manager with the plan.
> > It would be really good if Arun continues if this plan is adopted.
> > We can return to the RM topic if not.
> > 
> > The vote will run for 7 days until next Wed, May 8th.
> > 
> > Thanks,
> > --Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Tsz Wo Sze <sz...@yahoo.com>]

Oops, the "If yes," below should "If no,".




________________________________
 From: Tsz Wo Sze <sz...@yahoo.com>
To: "general@hadoop.apache.org" <ge...@hadoop.apache.org> 
Sent: Saturday, May 4, 2013 3:21 PM
Subject: Re: [VOTE] Release plan for Hadoop 2.0.5
 

Then, would you still +1 without the addition?  If yes, you should -1 on this VOTE.
Tsz-Wo



________________________________
From: Konstantin Boudnik <co...@apache.org>
To: general@hadoop.apache.org 
Sent: Thursday, May 2, 2013 8:40 PM
Subject: Re: [VOTE] Release plan for Hadoop 2.0.5


+1 with one addition I'd like to make to the plan

  - making downstream testing a passing criteria for the release. I don't mean
    that Hadoop community needs to be responsible for testing of _all_
    downstream components. What I'd like to see is a practice we used in the
    2.0.4-alpha with Bigtop 0.6 where passage of the tests in the latter was
    a, perhaps unofficial, release criteria for the former.

I also agree with 2.0.5 versioning, because essentially, future stability of
the API - of freezing - is the solid base we all are looking for.

Cos

On Wed, May 01, 2013 at 12:53PM, Konstantin Shvachko wrote:
> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
> 
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
> 
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
> 
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
> 
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
> 
> The vote will run for 7 days until next Wed, May 8th.
> 
> Thanks,
> --Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Tsz Wo Sze <sz...@yahoo.com>]

Then, would you still +1 without the addition?  If yes, you should -1 on this VOTE.
Tsz-Wo



________________________________
 From: Konstantin Boudnik <co...@apache.org>
To: general@hadoop.apache.org 
Sent: Thursday, May 2, 2013 8:40 PM
Subject: Re: [VOTE] Release plan for Hadoop 2.0.5
 

+1 with one addition I'd like to make to the plan

  - making downstream testing a passing criteria for the release. I don't mean
    that Hadoop community needs to be responsible for testing of _all_
    downstream components. What I'd like to see is a practice we used in the
    2.0.4-alpha with Bigtop 0.6 where passage of the tests in the latter was
    a, perhaps unofficial, release criteria for the former.

I also agree with 2.0.5 versioning, because essentially, future stability of
the API - of freezing - is the solid base we all are looking for.

Cos

On Wed, May 01, 2013 at 12:53PM, Konstantin Shvachko wrote:
> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
> 
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
> 
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
> 
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
> 
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
> 
> The vote will run for 7 days until next Wed, May 8th.
> 
> Thanks,
> --Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Boudnik <co...@apache.org>]

+1 with one addition I'd like to make to the plan

  - making downstream testing a passing criteria for the release. I don't mean
    that Hadoop community needs to be responsible for testing of _all_
    downstream components. What I'd like to see is a practice we used in the
    2.0.4-alpha with Bigtop 0.6 where passage of the tests in the latter was
    a, perhaps unofficial, release criteria for the former.

I also agree with 2.0.5 versioning, because essentially, future stability of
the API - of freezing - is the solid base we all are looking for.

Cos

On Wed, May 01, 2013 at 12:53PM, Konstantin Shvachko wrote:
> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
> 
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
> 
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
> 
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
> 
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
> 
> The vote will run for 7 days until next Wed, May 8th.
> 
> Thanks,
> --Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Andrew Purtell <ap...@apache.org>]

Making 2.0.x stable is really important for adoption of it as the new
production quality Hadoop.

I think Hadoop should adopt the merge window concept. After a release, a
new branch is opened for development. There is a window of time when
changes can go in, then the window is closed, then the branch is
stabilized, then the next major release is made off of the stabilized
branch. Repeat. What constitutes a major release numbering transition could
be from e.g. 2.x to 3.x, or from e.g. 2.0.x to 2.1.x. Either would work. If
feature development misses a window, it waits for the next. That is
completely fair and predictable. If the periodicity of open merge windows
is also predictable that is even better. If Hadoop can consider the merge
window now closed for 2.0.x, that would be great, it has been open for a
very long time.

+1 (nonbinding) after all of that, apologies for the verbiage but the
stability of the Hadoop ecosystem's foundation is a very important topic.
If I had a binding vote I would have voted +0 so as not to be presumptuous,
since my vote is nonbinding please consider it user feedback. Every single
project downstream is affected. We don't have a binding stake in the voting
but please consider us in your deliberations.

IMHO, a 2.1.0 branch and short merge window for getting HDFS snapshots out
there should quickly follow, and then the necessary 3-6 months of scale
testing. By all accounts talking with users it is the remaining compelling
feature missing from Hadoop 2.


On Fri, May 10, 2013 at 10:58 AM, lohit <lo...@gmail.com> wrote:

> +1
> Particularly making 2.0.X stable will help wider adoption sooner.
>
>
> 2013/5/9 J. Rottinghuis <jr...@gmail.com>
>
> > +1 (non-binding) to stabilize 2.0, and add new features only after a
> stable
> > release.
> > I understand that what constitutes as "new feature" versus "stabilizing"
> is
> > subjective.
> >
> > Thanks,
> >
> > Joep
> >
> > On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:
> > >
> > > > Please vote on the following plan for Hadoop release 2.0.5
> > > > - bug fixes encountered in current release 2.0.4-alpha
> > > > - make all API changes to allow freezing them post 2.0.5
> > > > - no new features
> > > >
> > > > As discussed on @dev thread
> > > > http://s.apache.org/fs
> > > > this will allow to stabilize 2.0 branch in a short and predictable
> > period
> > > > of time.
> > > > This enables a powerful option to have the release tested at Yahoo
> > scale.
> > > > The plan is to follow up with 2.1.0 - the stable release.
> > > > New features can and should be added on top of the stable release
> once
> > it
> > > > is out.
> > > >
> > > > Hadoop by-laws:
> > > > http://hadoop.apache.org/bylaws.html
> > > >
> > > > "Release Plan
> > > > Defines the timetable and actions for a release. The plan also
> > nominates
> > > a
> > > > Release Manager.
> > > > Lazy majority of active committers"
> > > >
> > > > assume nomination of a Release Manager with the plan.
> > > > It would be really good if Arun continues if this plan is adopted.
> > > > We can return to the RM topic if not.
> > > >
> > > > The vote will run for 7 days until next Wed, May 8th.
> > > >
> > > > Thanks,
> > > > --Konstantin
> > >
> > >
> >
>
>
>
> --
> Have a Nice Day!
> Lohit
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [VOTE] Release plan for Hadoop 2.0.5

[lohit <lo...@gmail.com>]

+1
Particularly making 2.0.X stable will help wider adoption sooner.


2013/5/9 J. Rottinghuis <jr...@gmail.com>

> +1 (non-binding) to stabilize 2.0, and add new features only after a stable
> release.
> I understand that what constitutes as "new feature" versus "stabilizing" is
> subjective.
>
> Thanks,
>
> Joep
>
> On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:
> >
> > > Please vote on the following plan for Hadoop release 2.0.5
> > > - bug fixes encountered in current release 2.0.4-alpha
> > > - make all API changes to allow freezing them post 2.0.5
> > > - no new features
> > >
> > > As discussed on @dev thread
> > > http://s.apache.org/fs
> > > this will allow to stabilize 2.0 branch in a short and predictable
> period
> > > of time.
> > > This enables a powerful option to have the release tested at Yahoo
> scale.
> > > The plan is to follow up with 2.1.0 - the stable release.
> > > New features can and should be added on top of the stable release once
> it
> > > is out.
> > >
> > > Hadoop by-laws:
> > > http://hadoop.apache.org/bylaws.html
> > >
> > > "Release Plan
> > > Defines the timetable and actions for a release. The plan also
> nominates
> > a
> > > Release Manager.
> > > Lazy majority of active committers"
> > >
> > > assume nomination of a Release Manager with the plan.
> > > It would be really good if Arun continues if this plan is adopted.
> > > We can return to the RM topic if not.
> > >
> > > The vote will run for 7 days until next Wed, May 8th.
> > >
> > > Thanks,
> > > --Konstantin
> >
> >
>



-- 
Have a Nice Day!
Lohit

Re: [VOTE] Release plan for Hadoop 2.0.5

["J. Rottinghuis" <jr...@gmail.com>]

+1 (non-binding) to stabilize 2.0, and add new features only after a stable
release.
I understand that what constitutes as "new feature" versus "stabilizing" is
subjective.

Thanks,

Joep

On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:
>
> > Please vote on the following plan for Hadoop release 2.0.5
> > - bug fixes encountered in current release 2.0.4-alpha
> > - make all API changes to allow freezing them post 2.0.5
> > - no new features
> >
> > As discussed on @dev thread
> > http://s.apache.org/fs
> > this will allow to stabilize 2.0 branch in a short and predictable period
> > of time.
> > This enables a powerful option to have the release tested at Yahoo scale.
> > The plan is to follow up with 2.1.0 - the stable release.
> > New features can and should be added on top of the stable release once it
> > is out.
> >
> > Hadoop by-laws:
> > http://hadoop.apache.org/bylaws.html
> >
> > "Release Plan
> > Defines the timetable and actions for a release. The plan also nominates
> a
> > Release Manager.
> > Lazy majority of active committers"
> >
> > assume nomination of a Release Manager with the plan.
> > It would be really good if Arun continues if this plan is adopted.
> > We can return to the RM topic if not.
> >
> > The vote will run for 7 days until next Wed, May 8th.
> >
> > Thanks,
> > --Konstantin
>
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Vinod Kumar Vavilapalli <vi...@hortonworks.com>]

That's a problem, if it isn't in the vote as to what feature in progress is being proposed to be included and what is excluded, how can we expect to vote?

Thanks,
+Vinod

On May 9, 2013, at 11:39 PM, Konstantin Shvachko wrote:

> Vinod,
> 
> I did ask Arun for the set of features for the release, and did ask to put
> it on vote.
> It is deep in the discussion thread, but you can find it.
> 
> Thank you for voting,
> --Konst
> 
> 
> On Thu, May 9, 2013 at 5:25 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> 
>> 
>> Confused and thought this was dead as the discussion was happening in
>> parallel. I wish this vote was retracted while the discussion reaches some
>> conclusion.
>> 
>> My concerns:
>> - As you noted in the by-laws, this plan should also nominate the RM. Or
>> if Arun is doing this, he should accept this? And may be call the vote for
>> release plan himself? The bylaws aren't clear about this, but that only
>> seems natural - you manage a release and so you call for a vote on the
>> plan, no?
>> - It's not clear what happens of features that are half-way through. That
>> needs to be spelled out.
>> - Even otherwise, we can always add features without destabilizing the
>> current release at all if we can switch the feature off with one flag.
>> Arguable on a case-by-case basis but possible.
>> - Also,  it isn't always clear what a feature is and what isn't. For e.g,
>> the ResourceManager restart work in YARN can be called a feature or a bug
>> depending on the context.
>> 
>> In all the above cases, we should discuss on whether a feature can be
>> merged in or not on a case-by-case basis instead of a blanket no.
>> 
>> -1 (binding)
>> 
>> Thanks,
>> +Vinod
>> Side note: The bylaws repeatedly talk of "active committers" and "active
>> PMC members", it makes sense, but we should clarify that.
>> 
>> On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:
>> 
>>> Please vote on the following plan for Hadoop release 2.0.5
>>> - bug fixes encountered in current release 2.0.4-alpha
>>> - make all API changes to allow freezing them post 2.0.5
>>> - no new features
>>> 
>>> As discussed on @dev thread
>>> http://s.apache.org/fs
>>> this will allow to stabilize 2.0 branch in a short and predictable period
>>> of time.
>>> This enables a powerful option to have the release tested at Yahoo scale.
>>> The plan is to follow up with 2.1.0 - the stable release.
>>> New features can and should be added on top of the stable release once it
>>> is out.
>>> 
>>> Hadoop by-laws:
>>> http://hadoop.apache.org/bylaws.html
>>> 
>>> "Release Plan
>>> Defines the timetable and actions for a release. The plan also nominates
>> a
>>> Release Manager.
>>> Lazy majority of active committers"
>>> 
>>> assume nomination of a Release Manager with the plan.
>>> It would be really good if Arun continues if this plan is adopted.
>>> We can return to the RM topic if not.
>>> 
>>> The vote will run for 7 days until next Wed, May 8th.
>>> 
>>> Thanks,
>>> --Konstantin
>> 
>> 

Re: [VOTE] Release plan for Hadoop 2.0.5

[Konstantin Shvachko <sh...@gmail.com>]

Vinod,

I did ask Arun for the set of features for the release, and did ask to put
it on vote.
It is deep in the discussion thread, but you can find it.

Thank you for voting,
--Konst


On Thu, May 9, 2013 at 5:25 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:

>
> Confused and thought this was dead as the discussion was happening in
> parallel. I wish this vote was retracted while the discussion reaches some
> conclusion.
>
> My concerns:
> - As you noted in the by-laws, this plan should also nominate the RM. Or
> if Arun is doing this, he should accept this? And may be call the vote for
> release plan himself? The bylaws aren't clear about this, but that only
> seems natural - you manage a release and so you call for a vote on the
> plan, no?
> - It's not clear what happens of features that are half-way through. That
> needs to be spelled out.
> - Even otherwise, we can always add features without destabilizing the
> current release at all if we can switch the feature off with one flag.
> Arguable on a case-by-case basis but possible.
> - Also,  it isn't always clear what a feature is and what isn't. For e.g,
> the ResourceManager restart work in YARN can be called a feature or a bug
> depending on the context.
>
> In all the above cases, we should discuss on whether a feature can be
> merged in or not on a case-by-case basis instead of a blanket no.
>
> -1 (binding)
>
> Thanks,
> +Vinod
> Side note: The bylaws repeatedly talk of "active committers" and "active
> PMC members", it makes sense, but we should clarify that.
>
> On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:
>
> > Please vote on the following plan for Hadoop release 2.0.5
> > - bug fixes encountered in current release 2.0.4-alpha
> > - make all API changes to allow freezing them post 2.0.5
> > - no new features
> >
> > As discussed on @dev thread
> > http://s.apache.org/fs
> > this will allow to stabilize 2.0 branch in a short and predictable period
> > of time.
> > This enables a powerful option to have the release tested at Yahoo scale.
> > The plan is to follow up with 2.1.0 - the stable release.
> > New features can and should be added on top of the stable release once it
> > is out.
> >
> > Hadoop by-laws:
> > http://hadoop.apache.org/bylaws.html
> >
> > "Release Plan
> > Defines the timetable and actions for a release. The plan also nominates
> a
> > Release Manager.
> > Lazy majority of active committers"
> >
> > assume nomination of a Release Manager with the plan.
> > It would be really good if Arun continues if this plan is adopted.
> > We can return to the RM topic if not.
> >
> > The vote will run for 7 days until next Wed, May 8th.
> >
> > Thanks,
> > --Konstantin
>
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Vinod Kumar Vavilapalli <vi...@hortonworks.com>]

Confused and thought this was dead as the discussion was happening in parallel. I wish this vote was retracted while the discussion reaches some conclusion.

My concerns:
- As you noted in the by-laws, this plan should also nominate the RM. Or if Arun is doing this, he should accept this? And may be call the vote for release plan himself? The bylaws aren't clear about this, but that only seems natural - you manage a release and so you call for a vote on the plan, no?
- It's not clear what happens of features that are half-way through. That needs to be spelled out.
- Even otherwise, we can always add features without destabilizing the current release at all if we can switch the feature off with one flag. Arguable on a case-by-case basis but possible. 
- Also,  it isn't always clear what a feature is and what isn't. For e.g, the ResourceManager restart work in YARN can be called a feature or a bug depending on the context.

In all the above cases, we should discuss on whether a feature can be merged in or not on a case-by-case basis instead of a blanket no.

-1 (binding)

Thanks,
+Vinod
Side note: The bylaws repeatedly talk of "active committers" and "active PMC members", it makes sense, but we should clarify that.

On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:

> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
> 
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
> 
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
> 
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
> 
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
> 
> The vote will run for 7 days until next Wed, May 8th.
> 
> Thanks,
> --Konstantin

Re: [VOTE] Release plan for Hadoop 2.0.5

[Eli Collins <el...@cloudera.com>]

On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features

Hey Konstantin,

It might make sense to have a separate discussion thread on this point
where you outline how you propose to make all API changes in 2.0.x
ahead of 2.1.0.  It seems like the only way you could do this w/o
having 2.0 and 2.1 diverge is to merge the relevant API changes from
branch-2 (ie that will be in 2.1) to the branch you create. However,
that means the API stabilization in your branch is gated by the 2.1.0
release anyway, and it's often hard to merge API changes w/o also
taking the feature they were introduced in.

It sounds like Yahoo! and others would benefit from a 2.0.x release
series where the existing 2.0.4 scope is stabilized, ie your 1st and
3rd bullets above, but I don't think the 2nd bullet makes sense
because the RM for the 2.0.x series doesn't get to make the call that
the 2.1.x APIs are frozen.  Also, as a community we're going to have
to make sure there's a good upgrade story around APIs from 0.23.x and
2.0.x to 2.1.x anyway.

Thanks,
Eli

Re: [VOTE] Release plan for Hadoop 2.0.5

[Chris Douglas <cd...@apache.org>]

-1 (binding) -C

On Wednesday, May 1, 2013, Konstantin Shvachko wrote:

> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
>
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
>
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
>
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
>
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
>
> The vote will run for 7 days until next Wed, May 8th.
>
> Thanks,
> --Konstantin
>

Re: [VOTE] Release plan for Hadoop 2.0.5

[Arun C Murthy <ac...@hortonworks.com>]

Konstantin, I think this is a little premature. 

Please see the continuing discussion on *-dev lists.

thanks,
Arun

On May 1, 2013, at 12:53 PM, Konstantin Shvachko wrote:

> Please vote on the following plan for Hadoop release 2.0.5
> - bug fixes encountered in current release 2.0.4-alpha
> - make all API changes to allow freezing them post 2.0.5
> - no new features
> 
> As discussed on @dev thread
> http://s.apache.org/fs
> this will allow to stabilize 2.0 branch in a short and predictable period
> of time.
> This enables a powerful option to have the release tested at Yahoo scale.
> The plan is to follow up with 2.1.0 - the stable release.
> New features can and should be added on top of the stable release once it
> is out.
> 
> Hadoop by-laws:
> http://hadoop.apache.org/bylaws.html
> 
> "Release Plan
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> Lazy majority of active committers"
> 
> assume nomination of a Release Manager with the plan.
> It would be really good if Arun continues if this plan is adopted.
> We can return to the RM topic if not.
> 
> The vote will run for 7 days until next Wed, May 8th.
> 
> Thanks,
> --Konstantin

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

Re: [VOTE] Release plan for Hadoop 2.0.5

[Roman Shaposhnik <rv...@apache.org>]

On Wed, May 1, 2013 at 12:53 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Please vote on the following plan for Hadoop release 2.0.5

I'm -1 (non-binding) on calling it 2.0.5. It has to be either 2.0.5-alpha
or 2.0.5-beta.

Thanks,
Roman.

P.S. For completeness' sake:
   # I'd be +1 (non-binding) on the content of this proposal for 2.0.5-alpha.
   # I'd be a strong -1 (non-binding ) on the content of this proposal
for 2.0.5-beta.
I believe that anything that has 'beta' in its name *has* to have the feedback
of downstream components be part of any formal release plan. IOW,
one (or all) of the following has to be explicitly mentioned and agreed upon:
    http://s.apache.org/sU

Re: [VOTE] Release plan for Hadoop 2.0.5

[Robert Evans <ev...@yahoo-inc.com>]

+1 (Binding)

--Bobby

On 5/1/13 2:53 PM, "Konstantin Shvachko" <sh...@gmail.com> wrote:

>Please vote on the following plan for Hadoop release 2.0.5
>- bug fixes encountered in current release 2.0.4-alpha
>- make all API changes to allow freezing them post 2.0.5
>- no new features
>
>As discussed on @dev thread
>http://s.apache.org/fs
>this will allow to stabilize 2.0 branch in a short and predictable period
>of time.
>This enables a powerful option to have the release tested at Yahoo scale.
>The plan is to follow up with 2.1.0 - the stable release.
>New features can and should be added on top of the stable release once it
>is out.
>
>Hadoop by-laws:
>http://hadoop.apache.org/bylaws.html
>
>"Release Plan
>Defines the timetable and actions for a release. The plan also nominates a
>Release Manager.
>Lazy majority of active committers"
>
>assume nomination of a Release Manager with the plan.
>It would be really good if Arun continues if this plan is adopted.
>We can return to the RM topic if not.
>
>The vote will run for 7 days until next Wed, May 8th.
>
>Thanks,
>--Konstantin