You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Niranjan Balasubramanian <ni...@cs.washington.edu> on 2011/11/22 21:05:07 UTC

Hadoop next stable release

Hello

We are currently using hadoop 0.20.203 on a 10 node cluster. We are considering upgrading to a newer version and I have two questions in this regard.

1) It seems 0.21 is unlikely to become a stable release anytime soon and we are weary of moving to an unstable release. Our primary concern is the data we have on our hdfs.  I want to know if anyone has been using 0.21 in production and would like to hear about your experiences? Any advice on this front is appreciated. 

2) Do we know when 0.23 is likely to become stable? There has been some discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it going to happen by the end of this year? 
 
Thanks
~ Niranjan.

* - http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+

Re: Hadoop next stable release

Posted by Konstantin Boudnik <co...@apache.org>.
We are expecting to release 0.22 very shortly. 0.22 is suppose to be
considered stable because it has been heavily tested at scale by eBay team
(as far as I know). However, I will let 0.22's RM to comment on that.

Cos

On Tue, Nov 22, 2011 at 12:05PM, Niranjan Balasubramanian wrote:
> Hello
> 
> We are currently using hadoop 0.20.203 on a 10 node cluster. We are
> considering upgrading to a newer version and I have two questions in this
> regard.
> 
> 1) It seems 0.21 is unlikely to become a stable release anytime soon and we
> are weary of moving to an unstable release. Our primary concern is the data
> we have on our hdfs.  I want to know if anyone has been using 0.21 in
> production and would like to hear about your experiences? Any advice on this
> front is appreciated. 
> 
> 2) Do we know when 0.23 is likely to become stable? There has been some
> discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
> going to happen by the end of this year? 
>  
> Thanks
> ~ Niranjan.
> 
> * - http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+

Re: Hadoop next stable release

Posted by Arun C Murthy <ac...@hortonworks.com>.
On Nov 22, 2011, at 12:05 PM, Niranjan Balasubramanian wrote:

> Hello
> 
> We are currently using hadoop 0.20.203 on a 10 node cluster. We are considering upgrading to a newer version and I have two questions in this regard.
> 
> 1) It seems 0.21 is unlikely to become a stable release anytime soon and we are weary of moving to an unstable release. Our primary concern is the data we have on our hdfs.  I want to know if anyone has been using 0.21 in production and would like to hear about your experiences? Any advice on this front is appreciated. 
> 
> 2) Do we know when 0.23 is likely to become stable? There has been some discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it going to happen by the end of this year? 
> 

Currently hadoop-0.23 has undergone significant feature, scale and performance testing at ~500 nodes, but there is some more to go.

We anticipate deploying to a few alpha clusters of sizes between 1000-2000 nodes in Jan in secure environments (is security important to you?).

I'd say for your scale late Q1, 2012 should a good time frame.

Arun


Re: Hadoop next stable release

Posted by Arun C Murthy <ac...@hortonworks.com>.
Sridhar,

On Nov 22, 2011, at 2:33 PM, Niranjan Balasubramanian wrote:

> Cos, Arun and Sridhar,
> 
> Thanks for the fast responses. Our two main requirements are:
> 
> 1. Multiple output -- Write multiple output files in each reducer. 
> 
> 2. Max Reducers per user -- Use Fairscheduler 0.21 or greater which would let us limit the maximum number of concurrent reducers per pool/user. 
> 

Those are fairly small reasons to upgrade... :)

You can easily use the MultipleOutput stuff with 0.20.2xx (it's just user-land library code) and FairScheduler changes are already in 0.20.205. You might want to consider that.

Arun


Re: Hadoop next stable release

Posted by Niranjan Balasubramanian <ni...@cs.washington.edu>.
Cos, Arun and Sridhar,

Thanks for the fast responses. Our two main requirements are:

1. Multiple output -- Write multiple output files in each reducer. 

2. Max Reducers per user -- Use Fairscheduler 0.21 or greater which would let us limit the maximum number of concurrent reducers per pool/user. 


~ Niranjan.

On Nov 22, 2011, at 2:19 PM, sridhar basam wrote:

> On Tue, Nov 22, 2011 at 3:05 PM, Niranjan Balasubramanian <
> niranjan@cs.washington.edu> wrote:
> 
>> Hello
>> 
>> We are currently using hadoop 0.20.203 on a 10 node cluster. We are
>> considering upgrading to a newer version and I have two questions in this
>> regard.
>> 
>> 1) It seems 0.21 is unlikely to become a stable release anytime soon and
>> we are weary of moving to an unstable release. Our primary concern is the
>> data we have on our hdfs.  I want to know if anyone has been using 0.21 in
>> production and would like to hear about your experiences? Any advice on
>> this front is appreciated.
>> 
>> 
> What new features or what deficiencies in the existing release are your
> trying to overcome. If primary concern is stability, wouldn't it be better
> to wait till 0.23 is GA'd. I don't think there are too many people running
> 0.21 at scale.
> 
> Sridhar
> 
> 
> 
>> 2) Do we know when 0.23 is likely to become stable? There has been some
>> discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
>> going to happen by the end of this year?
>> 
>> Thanks
>> ~ Niranjan.
>> 
>> * -
>> http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+


Re: Hadoop next stable release

Posted by sridhar basam <sr...@basam.org>.
On Tue, Nov 22, 2011 at 3:05 PM, Niranjan Balasubramanian <
niranjan@cs.washington.edu> wrote:

> Hello
>
> We are currently using hadoop 0.20.203 on a 10 node cluster. We are
> considering upgrading to a newer version and I have two questions in this
> regard.
>
> 1) It seems 0.21 is unlikely to become a stable release anytime soon and
> we are weary of moving to an unstable release. Our primary concern is the
> data we have on our hdfs.  I want to know if anyone has been using 0.21 in
> production and would like to hear about your experiences? Any advice on
> this front is appreciated.
>
>
What new features or what deficiencies in the existing release are your
trying to overcome. If primary concern is stability, wouldn't it be better
to wait till 0.23 is GA'd. I don't think there are too many people running
0.21 at scale.

 Sridhar



> 2) Do we know when 0.23 is likely to become stable? There has been some
> discussion on mail #dev* about 0.23 becoming stable sometime soon. Is it
> going to happen by the end of this year?
>
> Thanks
> ~ Niranjan.
>
> * -
> http://search-hadoop.com/m/f623FA7bDK1/hadoop+next+stable+release+0.21&subj=0+21+stable+schedule+