You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Konstantin Shvachko <sh...@gmail.com> on 2011/11/29 11:47:58 UTC
[VOTE] Release hadoop-0.22.0-rc0
I created a release candidate for hadoop-0.22.0 available for review at:
http://people.apache.org/~shv/hadoop-0.22.0-rc0/
The candidate incorporates more than 700 fixed jiras counting from the
release of 0.21.0.
The branch was cut on November 17 last year. During this year the
branch was tested by different groups in the Hadoop community.
Recently it has been extensively tested on a mid-size cluster.
It contains important improvements, optimizations and bug fixes for
HDFS and MapReduce compared to 0.21.0. Some features introduced in the
latest offspring releases of hadoop-security are not yet ported to
this branch, which is mostly related to security and MapReduce
improvements. Compared to 0.23.0 this release will not contain
Federation or Yarn.
The hadoop-0.22.0 release build is fully automated thanks to the
BigTop team. rc0 artifacts are available and reproducible on Hadoop
Jenkins. The validity of rc0 has been also verified with BigTop.
I think this could be a usable release and an important step forward
in the Hadoop roadmap.
Many people contributed to hadoop-0.22. Thank you everybody who
developed code, built, tested, found and fixed bugs or supported this
effort by any other means.
Please try it out and vote.
Thanks,
--Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
On Fri, Dec 2, 2011 at 6:48 PM, Eric Baldeschwieler
<er...@hortonworks.com> wrote:
> I'm fine with the idea of alpha releases. I think they are important. I'm a bit confused on where this is going.
>
> what are your criteria for declaring the 0.22 branch stable?
I am not declaring anything. I am just proposing a release, which a
part of the community worked on and which runs on several clusters
that I can observe.
It is up to the community to decide what is stable and what is not.
> Is compatibility with 1.0 and security a goal for this release line?
As previously discussed security is turned off in this release.
LinuxTaskController has been removed.
I don't know what you mean by compatibility with 1.0.
--Konstantin
> ---
> E14 - typing on glass
>
> On Dec 2, 2011, at 1:15 AM, Konstantin Shvachko <sh...@gmail.com> wrote:
>
>> This is a well tested state of the branch, which imo worth releasing.
>> Adding changes at this point will invalidate the testing.
>> 0.23 has "substantial regressions from the stable line" as well, and
>> is not stable.
>> 1.0 does not have important features including some in MR that 0.22 has.
>> What is a regression!
>>
>> 0.22 is progress in at least one area, which is HDFS.
>> People can actually start appending data to files, use links, and
>> build features (e.g. HA) based on BackupNode.
>> HBase clusters should not be segregated anymore as HBase-only ones,
>> but will allow general load as well.
>>
>> Speaking of confusing. What isn't in Hadoop this days. Say, when you
>> transition from Hadoop 1.0 to 0.22 or 0.23 you call it upgrade.
>>
>> Seriously. This project needs to move forward beyond the 0.20 line. I
>> tested 0.22 in the last three months and found it working with HBase,
>> Hive, Pig, Cascading, Zookeeper, Oozie. What else do you need to
>> release? It is not perfect? Agreed. Have anybody seen a perfect piece
>> of software? I haven't.
>>
>> Eric, thank you for raising the question. It is important to
>> understand the pros and cons of different versions.
>> You should try the candidate, who knows it might just work for you.
>>
>> --Konstantin
>>
>> On Thu, Dec 1, 2011 at 10:26 PM, Eric Baldeschwieler
>> <er...@hortonworks.com> wrote:
>>> So this release contains substantial regressions from the stable line and contains a subset of features in 0.23.
>>>
>>> Wouldn't it be prudent to wait until the branch is compelling when compared to one of the other release lines before releasing? Otherwise this just seems confusing.
>>>
>>> ---
>>> E14 - typing on glass
>>>
>>> On Dec 1, 2011, at 6:10 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>>
>>>> +1
>>>>
>>>> Checksum are correct, license is in place
>>>> Single node cluster (Ubuntu 11.04) installation and a mapreduce job worked
>>>> without any issues.
>>>>
>>>> Cos
>>>>
>>>> On Tue, Nov 29, 2011 at 02:47AM, Konstantin Shvachko wrote:
>>>>> I created a release candidate for hadoop-0.22.0 available for review at:
>>>>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>>>>
>>>>> The candidate incorporates more than 700 fixed jiras counting from the
>>>>> release of 0.21.0.
>>>>> The branch was cut on November 17 last year. During this year the
>>>>> branch was tested by different groups in the Hadoop community.
>>>>> Recently it has been extensively tested on a mid-size cluster.
>>>>> It contains important improvements, optimizations and bug fixes for
>>>>> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
>>>>> latest offspring releases of hadoop-security are not yet ported to
>>>>> this branch, which is mostly related to security and MapReduce
>>>>> improvements. Compared to 0.23.0 this release will not contain
>>>>> Federation or Yarn.
>>>>> The hadoop-0.22.0 release build is fully automated thanks to the
>>>>> BigTop team. rc0 artifacts are available and reproducible on Hadoop
>>>>> Jenkins. The validity of rc0 has been also verified with BigTop.
>>>>> I think this could be a usable release and an important step forward
>>>>> in the Hadoop roadmap.
>>>>>
>>>>> Many people contributed to hadoop-0.22. Thank you everybody who
>>>>> developed code, built, tested, found and fixed bugs or supported this
>>>>> effort by any other means.
>>>>>
>>>>> Please try it out and vote.
>>>>>
>>>>> Thanks,
>>>>> --Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Eric Baldeschwieler <er...@hortonworks.com>.
I'm fine with the idea of alpha releases. I think they are important. I'm a bit confused on where this is going.
what are your criteria for declaring the 0.22 branch stable? Is compatibility with 1.0 and security a goal for this release line?
---
E14 - typing on glass
On Dec 2, 2011, at 1:15 AM, Konstantin Shvachko <sh...@gmail.com> wrote:
> This is a well tested state of the branch, which imo worth releasing.
> Adding changes at this point will invalidate the testing.
> 0.23 has "substantial regressions from the stable line" as well, and
> is not stable.
> 1.0 does not have important features including some in MR that 0.22 has.
> What is a regression!
>
> 0.22 is progress in at least one area, which is HDFS.
> People can actually start appending data to files, use links, and
> build features (e.g. HA) based on BackupNode.
> HBase clusters should not be segregated anymore as HBase-only ones,
> but will allow general load as well.
>
> Speaking of confusing. What isn't in Hadoop this days. Say, when you
> transition from Hadoop 1.0 to 0.22 or 0.23 you call it upgrade.
>
> Seriously. This project needs to move forward beyond the 0.20 line. I
> tested 0.22 in the last three months and found it working with HBase,
> Hive, Pig, Cascading, Zookeeper, Oozie. What else do you need to
> release? It is not perfect? Agreed. Have anybody seen a perfect piece
> of software? I haven't.
>
> Eric, thank you for raising the question. It is important to
> understand the pros and cons of different versions.
> You should try the candidate, who knows it might just work for you.
>
> --Konstantin
>
> On Thu, Dec 1, 2011 at 10:26 PM, Eric Baldeschwieler
> <er...@hortonworks.com> wrote:
>> So this release contains substantial regressions from the stable line and contains a subset of features in 0.23.
>>
>> Wouldn't it be prudent to wait until the branch is compelling when compared to one of the other release lines before releasing? Otherwise this just seems confusing.
>>
>> ---
>> E14 - typing on glass
>>
>> On Dec 1, 2011, at 6:10 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>>> +1
>>>
>>> Checksum are correct, license is in place
>>> Single node cluster (Ubuntu 11.04) installation and a mapreduce job worked
>>> without any issues.
>>>
>>> Cos
>>>
>>> On Tue, Nov 29, 2011 at 02:47AM, Konstantin Shvachko wrote:
>>>> I created a release candidate for hadoop-0.22.0 available for review at:
>>>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>>>
>>>> The candidate incorporates more than 700 fixed jiras counting from the
>>>> release of 0.21.0.
>>>> The branch was cut on November 17 last year. During this year the
>>>> branch was tested by different groups in the Hadoop community.
>>>> Recently it has been extensively tested on a mid-size cluster.
>>>> It contains important improvements, optimizations and bug fixes for
>>>> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
>>>> latest offspring releases of hadoop-security are not yet ported to
>>>> this branch, which is mostly related to security and MapReduce
>>>> improvements. Compared to 0.23.0 this release will not contain
>>>> Federation or Yarn.
>>>> The hadoop-0.22.0 release build is fully automated thanks to the
>>>> BigTop team. rc0 artifacts are available and reproducible on Hadoop
>>>> Jenkins. The validity of rc0 has been also verified with BigTop.
>>>> I think this could be a usable release and an important step forward
>>>> in the Hadoop roadmap.
>>>>
>>>> Many people contributed to hadoop-0.22. Thank you everybody who
>>>> developed code, built, tested, found and fixed bugs or supported this
>>>> effort by any other means.
>>>>
>>>> Please try it out and vote.
>>>>
>>>> Thanks,
>>>> --Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
> Given, the 'feature' regressions in MR (security, operability etc. - http://s.apache.org/critical-0.22.0), I propose we mark hadoop-0.22.0 as 'alpha' quality too with necessary statutory warnings to end users to alert them to the differences in terms of missing features. Else, we could land up confusing users - this has come up before on this list: http://s.apache.org/Go9. Seems reasonable?
>
Yes the community can decide to qualify hadoop-0.22.0 as 'alpha'. As
RM I only propose a release candidate, everybody else can evaluate the
quality. We can have a discussion in a separate thread.
Based on my personal experience I would not qualify it as "not yet
ready for serious use".
> On a related note - it would help if you could share your thoughts on the roadmap for releases off branch-0.22.
> Some questions come to mind, these would help the community (and, of course, end-users) evaluate current & future releases of the branch:
> # Do you plan to fix security in branch-0.22?
> # Do you plan to _test_ security end-to-end for the whole ecosystem?
> # Do you plan to fix some of the critical operability features such as disk-fail-in-place, MR tasklogs' handling etc.?
> # Could you share any details on the performance work/benchmarks you've done for 0.22? How does it compare to hadoop-0.20.2xx for e.g.? Various folks, at different points, have noticed performance regressions in branch-0.22 - have you checked?
> # Do you have timelines in mind for the above?
These are all good questions Arun. All of these items would improve
the quality of the branch.
Again, I don't see RM's role as a spiritual leader drawing roadmaps
for the community. We all should decide where we want to go with this
and other releases.
I can say for myself that I will be closely working with this branch
for at least some time. And as usually will make sure to merge all
findings into it.
As a discussion topic (for a separate thread) we can consider two
options for upgrading MR in 0.22:
1) To do forward porting of improvements from 1.0.
2) To merge MR-1.0 into branch-0.22 and make it work with HDFS.22
Ideally, it would be more productive to converge under a single
branch, rather than spreading the development resources among multiple
ones. Yet another way is to have a higher level of compatibility
between versions/branches, so that one could mix and match components,
say HDFS.22 + MR.23.
I heard other people thinking along the same lines.
Thanks,
--Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Arun C Murthy <ac...@hortonworks.com>.
Konstantin,
On Dec 3, 2011, at 4:43 PM, Konstantin Shvachko wrote:
>
> On Fri, Dec 2, 2011 at 9:39 PM, Arun Murthy <ac...@hortonworks.com> wrote:
>> On Dec 2, 2011, at 1:16 AM, Konstantin Shvachko <sh...@gmail.com> wrote:
>>
>>> 0.23 has "substantial regressions from the stable line" as well, and
>>> is not stable.
>>
>> Do you have a list Konstantin? We'll shoot to fix them for 0.23.1...
>>
>
> This was not intended to criticize 0.23 release.
> I am merely referring to its alpha status.
> IMO 0.23 is a good and timely release.
No offense taken, we've been very clear about 'production-worthiness' of hadoop-0.23.0 at this point in time.
For example, here is the warning for end-users on the releases page:
http://hadoop.apache.org/common/releases.html#11+Nov%2C+2011%3A+release+0.23.0+available
OTOH, if you are aware of any 'feature regressions' please bring it to our collective attention and we'll try to get them fixed for 0.23.1. One feature regression I knew of i.e. disk-fail-in-place for the NodeManager is already fixed for 0.23.1 (MAPREDUCE-3121). If you have others, please let me know.
Also, we've agreed, on this list, that hadoop-0.23.1 (sometime in mid-to-late December) should be marked as 'alpha' quality as we continue to make progress stabilizing branch-0.23.
Given, the 'feature' regressions in MR (security, operability etc. - http://s.apache.org/critical-0.22.0), I propose we mark hadoop-0.22.0 as 'alpha' quality too with necessary statutory warnings to end users to alert them to the differences in terms of missing features. Else, we could land up confusing users - this has come up before on this list: http://s.apache.org/Go9. Seems reasonable?
On a related note - it would help if you could share your thoughts on the roadmap for releases off branch-0.22.
Some questions come to mind, these would help the community (and, of course, end-users) evaluate current & future releases of the branch:
# Do you plan to fix security in branch-0.22?
# Do you plan to _test_ security end-to-end for the whole ecosystem?
# Do you plan to fix some of the critical operability features such as disk-fail-in-place, MR tasklogs' handling etc.?
# Could you share any details on the performance work/benchmarks you've done for 0.22? How does it compare to hadoop-0.20.2xx for e.g.? Various folks, at different points, have noticed performance regressions in branch-0.22 - have you checked?
# Do you have timelines in mind for the above?
thanks,
Arun
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Arun,
This was not intended to criticize 0.23 release.
I am merely referring to its alpha status.
IMO 0.23 is a good and timely release.
Thanks,
--Konstantin
On Fri, Dec 2, 2011 at 9:39 PM, Arun Murthy <ac...@hortonworks.com> wrote:
> On Dec 2, 2011, at 1:16 AM, Konstantin Shvachko <sh...@gmail.com> wrote:
>
>> 0.23 has "substantial regressions from the stable line" as well, and
>> is not stable.
>
> Do you have a list Konstantin? We'll shoot to fix them for 0.23.1...
>
> thanks,
> Arun
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Arun Murthy <ac...@hortonworks.com>.
On Dec 2, 2011, at 1:16 AM, Konstantin Shvachko <sh...@gmail.com> wrote:
> 0.23 has "substantial regressions from the stable line" as well, and
> is not stable.
Do you have a list Konstantin? We'll shoot to fix them for 0.23.1...
thanks,
Arun
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Jakob Homan <jg...@gmail.com>.
+1. Downloaded, verified, started cluster, poked and prodded. Quite
happy about this.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
This is a well tested state of the branch, which imo worth releasing.
Adding changes at this point will invalidate the testing.
0.23 has "substantial regressions from the stable line" as well, and
is not stable.
1.0 does not have important features including some in MR that 0.22 has.
What is a regression!
0.22 is progress in at least one area, which is HDFS.
People can actually start appending data to files, use links, and
build features (e.g. HA) based on BackupNode.
HBase clusters should not be segregated anymore as HBase-only ones,
but will allow general load as well.
Speaking of confusing. What isn't in Hadoop this days. Say, when you
transition from Hadoop 1.0 to 0.22 or 0.23 you call it upgrade.
Seriously. This project needs to move forward beyond the 0.20 line. I
tested 0.22 in the last three months and found it working with HBase,
Hive, Pig, Cascading, Zookeeper, Oozie. What else do you need to
release? It is not perfect? Agreed. Have anybody seen a perfect piece
of software? I haven't.
Eric, thank you for raising the question. It is important to
understand the pros and cons of different versions.
You should try the candidate, who knows it might just work for you.
--Konstantin
On Thu, Dec 1, 2011 at 10:26 PM, Eric Baldeschwieler
<er...@hortonworks.com> wrote:
> So this release contains substantial regressions from the stable line and contains a subset of features in 0.23.
>
> Wouldn't it be prudent to wait until the branch is compelling when compared to one of the other release lines before releasing? Otherwise this just seems confusing.
>
> ---
> E14 - typing on glass
>
> On Dec 1, 2011, at 6:10 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
>> +1
>>
>> Checksum are correct, license is in place
>> Single node cluster (Ubuntu 11.04) installation and a mapreduce job worked
>> without any issues.
>>
>> Cos
>>
>> On Tue, Nov 29, 2011 at 02:47AM, Konstantin Shvachko wrote:
>>> I created a release candidate for hadoop-0.22.0 available for review at:
>>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>>
>>> The candidate incorporates more than 700 fixed jiras counting from the
>>> release of 0.21.0.
>>> The branch was cut on November 17 last year. During this year the
>>> branch was tested by different groups in the Hadoop community.
>>> Recently it has been extensively tested on a mid-size cluster.
>>> It contains important improvements, optimizations and bug fixes for
>>> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
>>> latest offspring releases of hadoop-security are not yet ported to
>>> this branch, which is mostly related to security and MapReduce
>>> improvements. Compared to 0.23.0 this release will not contain
>>> Federation or Yarn.
>>> The hadoop-0.22.0 release build is fully automated thanks to the
>>> BigTop team. rc0 artifacts are available and reproducible on Hadoop
>>> Jenkins. The validity of rc0 has been also verified with BigTop.
>>> I think this could be a usable release and an important step forward
>>> in the Hadoop roadmap.
>>>
>>> Many people contributed to hadoop-0.22. Thank you everybody who
>>> developed code, built, tested, found and fixed bugs or supported this
>>> effort by any other means.
>>>
>>> Please try it out and vote.
>>>
>>> Thanks,
>>> --Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Eric Baldeschwieler <er...@hortonworks.com>.
So this release contains substantial regressions from the stable line and contains a subset of features in 0.23.
Wouldn't it be prudent to wait until the branch is compelling when compared to one of the other release lines before releasing? Otherwise this just seems confusing.
---
E14 - typing on glass
On Dec 1, 2011, at 6:10 PM, Konstantin Boudnik <co...@apache.org> wrote:
> +1
>
> Checksum are correct, license is in place
> Single node cluster (Ubuntu 11.04) installation and a mapreduce job worked
> without any issues.
>
> Cos
>
> On Tue, Nov 29, 2011 at 02:47AM, Konstantin Shvachko wrote:
>> I created a release candidate for hadoop-0.22.0 available for review at:
>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>
>> The candidate incorporates more than 700 fixed jiras counting from the
>> release of 0.21.0.
>> The branch was cut on November 17 last year. During this year the
>> branch was tested by different groups in the Hadoop community.
>> Recently it has been extensively tested on a mid-size cluster.
>> It contains important improvements, optimizations and bug fixes for
>> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
>> latest offspring releases of hadoop-security are not yet ported to
>> this branch, which is mostly related to security and MapReduce
>> improvements. Compared to 0.23.0 this release will not contain
>> Federation or Yarn.
>> The hadoop-0.22.0 release build is fully automated thanks to the
>> BigTop team. rc0 artifacts are available and reproducible on Hadoop
>> Jenkins. The validity of rc0 has been also verified with BigTop.
>> I think this could be a usable release and an important step forward
>> in the Hadoop roadmap.
>>
>> Many people contributed to hadoop-0.22. Thank you everybody who
>> developed code, built, tested, found and fixed bugs or supported this
>> effort by any other means.
>>
>> Please try it out and vote.
>>
>> Thanks,
>> --Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <co...@apache.org>.
+1
Checksum are correct, license is in place
Single node cluster (Ubuntu 11.04) installation and a mapreduce job worked
without any issues.
Cos
On Tue, Nov 29, 2011 at 02:47AM, Konstantin Shvachko wrote:
> I created a release candidate for hadoop-0.22.0 available for review at:
> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>
> The candidate incorporates more than 700 fixed jiras counting from the
> release of 0.21.0.
> The branch was cut on November 17 last year. During this year the
> branch was tested by different groups in the Hadoop community.
> Recently it has been extensively tested on a mid-size cluster.
> It contains important improvements, optimizations and bug fixes for
> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
> latest offspring releases of hadoop-security are not yet ported to
> this branch, which is mostly related to security and MapReduce
> improvements. Compared to 0.23.0 this release will not contain
> Federation or Yarn.
> The hadoop-0.22.0 release build is fully automated thanks to the
> BigTop team. rc0 artifacts are available and reproducible on Hadoop
> Jenkins. The validity of rc0 has been also verified with BigTop.
> I think this could be a usable release and an important step forward
> in the Hadoop roadmap.
>
> Many people contributed to hadoop-0.22. Thank you everybody who
> developed code, built, tested, found and fixed bugs or supported this
> effort by any other means.
>
> Please try it out and vote.
>
> Thanks,
> --Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Tom White <to...@cloudera.com>.
+1
Checksums and signature are good. I ran some MapReduce jobs and they passed.
Cheers,
Tom
On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I created a release candidate for hadoop-0.22.0 available for review at:
> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>
> The candidate incorporates more than 700 fixed jiras counting from the
> release of 0.21.0.
> The branch was cut on November 17 last year. During this year the
> branch was tested by different groups in the Hadoop community.
> Recently it has been extensively tested on a mid-size cluster.
> It contains important improvements, optimizations and bug fixes for
> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
> latest offspring releases of hadoop-security are not yet ported to
> this branch, which is mostly related to security and MapReduce
> improvements. Compared to 0.23.0 this release will not contain
> Federation or Yarn.
> The hadoop-0.22.0 release build is fully automated thanks to the
> BigTop team. rc0 artifacts are available and reproducible on Hadoop
> Jenkins. The validity of rc0 has been also verified with BigTop.
> I think this could be a usable release and an important step forward
> in the Hadoop roadmap.
>
> Many people contributed to hadoop-0.22. Thank you everybody who
> developed code, built, tested, found and fixed bugs or supported this
> effort by any other means.
>
> Please try it out and vote.
>
> Thanks,
> --Konstantin
RE: [VOTE] Release hadoop-0.22.0-rc0
Posted by Uma Maheswara Rao G <ma...@huawei.com>.
+1 Downloaded the RC0 release. Executed all basic filesystem operations with 10 threads.
Also executed some internal mapred applications with out any issues.
________________________________________
From: Konstantin Shvachko [shv.hadoop@gmail.com]
Sent: Tuesday, November 29, 2011 4:17 PM
To: general@hadoop.apache.org
Subject: [VOTE] Release hadoop-0.22.0-rc0
I created a release candidate for hadoop-0.22.0 available for review at:
http://people.apache.org/~shv/hadoop-0.22.0-rc0/
The candidate incorporates more than 700 fixed jiras counting from the
release of 0.21.0.
The branch was cut on November 17 last year. During this year the
branch was tested by different groups in the Hadoop community.
Recently it has been extensively tested on a mid-size cluster.
It contains important improvements, optimizations and bug fixes for
HDFS and MapReduce compared to 0.21.0. Some features introduced in the
latest offspring releases of hadoop-security are not yet ported to
this branch, which is mostly related to security and MapReduce
improvements. Compared to 0.23.0 this release will not contain
Federation or Yarn.
The hadoop-0.22.0 release build is fully automated thanks to the
BigTop team. rc0 artifacts are available and reproducible on Hadoop
Jenkins. The validity of rc0 has been also verified with BigTop.
I think this could be a usable release and an important step forward
in the Hadoop roadmap.
Many people contributed to hadoop-0.22. Thank you everybody who
developed code, built, tested, found and fixed bugs or supported this
effort by any other means.
Please try it out and vote.
Thanks,
--Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Benoy Antony <ba...@gmail.com>.
+1 on releasing Hadoop 0.22
I deployed the 22 version , ran few tests for file and directory
permissions, Also tested space quotas.
They work fine.
Thanks ,
Benoy Antony.
On Mon, Dec 5, 2011 at 8:42 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sat, Dec 3, 2011 at 5:48 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
> > I updated the artifacts.
>
> +1 on releasing Hadoop 0.22
>
> I pulled the artifacts into Bigtop, verified the signatures and built
> packages for the
> following stack:
> Hadoop 0.22 RC
> HBase 0.92 RC
> Zookeeper 3.4.0
>
> The resulting packages are available from the usual place:
>
> http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/
>
> You can install them by DLing the repo file and following the Bigtop
> instructions:
>
> http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
>
> https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
>
> I have also deployed a fully distributed cluster of 5 nodes on Amazon
> EC2 and ran
> Bigtop validation testsuite:
>
> http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-smoketest/28/testReport/
>
> The only persistent failure has to do with new HBase functionality
> (testMRIncrementalLoadWithSplit).
>
> Thanks,
> Roman.
>
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Dec 3, 2011 at 5:48 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I updated the artifacts.
+1 on releasing Hadoop 0.22
I pulled the artifacts into Bigtop, verified the signatures and built
packages for the
following stack:
Hadoop 0.22 RC
HBase 0.92 RC
Zookeeper 3.4.0
The resulting packages are available from the usual place:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/
You can install them by DLing the repo file and following the Bigtop
instructions:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
I have also deployed a fully distributed cluster of 5 nodes on Amazon
EC2 and ran
Bigtop validation testsuite:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-smoketest/28/testReport/
The only persistent failure has to do with new HBase functionality
(testMRIncrementalLoadWithSplit).
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Owen O'Malley <ow...@hortonworks.com>.
On Fri, Dec 9, 2011 at 1:36 AM, J. Rottinghuis <jr...@gmail.com> wrote:
> Do we need a 0.22 version of the docs here?
> http://hadoop.apache.org/common/docs/
>
> Or is that only after a release candidate has been accepted by vote?
That is only after the vote passes.
-- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by "J. Rottinghuis" <jr...@gmail.com>.
Do we need a 0.22 version of the docs here?
http://hadoop.apache.org/common/docs/
Or is that only after a release candidate has been accepted by vote?
Thanks,
Joep
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by "J. Rottinghuis" <jr...@gmail.com>.
+1 This rc looks great!
Installed on production node with pretty much the same config (turned off
topology script and adjusted dfs.name.dir from a path to url).
Ran typical hadoop fs commands (copyFromLocal, copyToLocal, rm, rmr, cp,
du, etc)
Ran some of the examples (teragen, terasort, randomwriter, bbp)
Ran several distcp jobs using an hftp source url top copy various datasets
from clusters with different Hadoop version.
Ran a dozen or so jobs and killed jobs and task-attempts.
Ran hdfs fsck and dfsadmin
On the NN and JT UI I clicked through all the links I could find.
Observations:
When submitting jobs I see:
"WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated.
Instead, use mapreduce.client.genericoptionsparser.used"
When running distcp I see the additional warning:
"WARN conf.Configuration: mapred.job.name is deprecated. Instead, use
mapreduce.job.name"
When no TT can connect to the JT (due to missing topology script), clicking
on the number in the nodes column on jobtracker.jsp shows an exception. I'm
letting a larger distcp finish overnight, will reproduce error and file
bugs for the items above.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Stack <st...@duboce.net>.
+1
Ran an upgrade from tip of 205 branch on a 15 node cluster that was
carrying a bit of data. Then deployed tip of hbase 0.92 branch (built
against hadoop 0.22.0) on this hadoop-0.22.0RC and ran a basic upload
that used MR w/ a bit of node killing in-between for kicks (the sync
stuff looked to be working properly).
St.Ack
RE: [VOTE] Release hadoop-0.22.0-rc0
Posted by Uma Maheswara Rao G <ma...@huawei.com>.
+1
executed the same tests again with the updated artifacts, Working fine.
________________________________________
From: Konstantin Shvachko [shv.hadoop@gmail.com]
Sent: Sunday, December 04, 2011 7:18 AM
To: general@hadoop.apache.org
Subject: Re: [VOTE] Release hadoop-0.22.0-rc0
I updated the artifacts.
--Konstantin
On Sat, Dec 3, 2011 at 4:40 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Roman,
>
> Thanks for finding this.
> The change is actually in the assemble script in Bigtop, which should
> leave lib directories and the .txt files in the respective projects
> rather then moving them.
> I see you've already fixed the script. Thanks.
> I will upload the new artifact as soon as it built. There was a
> problem with disk space on ubuntu3.
>
> The question is should we reset the voting period.
> Since there was no change to the Hadoop tree I would rather not,
> unless people ask for it.
>
> Thanks,
> --Konstantin
>
> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>> <sh...@gmail.com> wrote:
>>> I created a release candidate for hadoop-0.22.0 available for review at:
>>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>
>> Pulled this RC into Bigtop and certified the following stack:
>> Hadoop 0.22
>> HBase 0.92
>> Zookeeper 3.4.0
>>
>> Found the following issues (which I suggest we fix and cut RC1):
>> 1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>> -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>> tar
>> [copy] Copying 48 files to
>> /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>>
>> BUILD FAILED
>> /tmp/hadoop-0.22.0/common/build.xml:1134:
>> /tmp/hadoop-0.22.0/common/lib not found.
>>
>> 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>> common, hdfs and mapred
>>
>> I will update Bigtop build scripts to take care of missing lib/
>> creation and also missing files.
>>
>> Once that's done we can cut a new RC when needed.
>>
>> Thanks,
>> Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <co...@apache.org>.
+1 - everything seems correct.
On Sat, Dec 03, 2011 at 05:48PM, Konstantin Shvachko wrote:
> I updated the artifacts.
> --Konstantin
>
> On Sat, Dec 3, 2011 at 4:40 PM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
> > Roman,
> >
> > Thanks for finding this.
> > The change is actually in the assemble script in Bigtop, which should
> > leave lib directories and the .txt files in the respective projects
> > rather then moving them.
> > I see you've already fixed the script. Thanks.
> > I will upload the new artifact as soon as it built. There was a
> > problem with disk space on ubuntu3.
> >
> > The question is should we reset the voting period.
> > Since there was no change to the Hadoop tree I would rather not,
> > unless people ask for it.
> >
> > Thanks,
> > --Konstantin
> >
> > On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> >> On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
> >> <sh...@gmail.com> wrote:
> >>> I created a release candidate for hadoop-0.22.0 available for review at:
> >>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
> >>
> >> Pulled this RC into Bigtop and certified the following stack:
> >> ═ Hadoop 0.22
> >> ═ HBase 0.92
> >> ═ Zookeeper 3.4.0
> >>
> >> Found the following issues (which I suggest we fix and cut RC1):
> >> ═1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
> >> -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
> >> tar
> >> ═ ═[copy] Copying 48 files to
> >> /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
> >>
> >> ═ ═BUILD FAILED
> >> ═ ═/tmp/hadoop-0.22.0/common/build.xml:1134:
> >> ═ ═/tmp/hadoop-0.22.0/common/lib not found.
> >>
> >> ═ 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
> >> common, hdfs and mapred
> >>
> >> I will update Bigtop build scripts to take care of missing lib/
> >> creation and also missing files.
> >>
> >> Once that's done we can cut a new RC when needed.
> >>
> >> Thanks,
> >> Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Dec 3, 2011 at 5:48 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I updated the artifacts.
+1 on releasing Hadoop 0.22
I pulled the artifacts into Bigtop, verified the signatures and built
packages for the
following stack:
Hadoop 0.22 RC
HBase 0.92 RC
Zookeeper 3.4.0
The resulting packages are available from the usual place:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/
You can install them by DLing the repo file and following the Bigtop
instructions:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
I have also deployed a fully distributed cluster of 5 nodes on Amazon
EC2 and ran
Bigtop validation testsuite:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-smoketest/28/testReport/
The only persistent failure has to do with new HBase functionality
(testMRIncrementalLoadWithSplit).
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Dec 3, 2011 at 5:48 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I updated the artifacts.
+1 on releasing Hadoop 0.22
I pulled the artifacts into Bigtop, verified the signatures and built
packages for the
following stack:
Hadoop 0.22 RC
HBase 0.92 RC
Zookeeper 3.4.0
The resulting packages are available from the usual place:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/
You can install them by DLing the repo file and following the Bigtop
instructions:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
https://cwiki.apache.org/confluence/display/BIGTOP/How+to+install+Hadoop+distribution+from+Bigtop
I have also deployed a fully distributed cluster of 5 nodes on Amazon
EC2 and ran
Bigtop validation testsuite:
http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-hadoop22-smoketest/28/testReport/
The only persistent failure has to do with new HBase functionality
(testMRIncrementalLoadWithSplit).
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
I updated the artifacts.
--Konstantin
On Sat, Dec 3, 2011 at 4:40 PM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Roman,
>
> Thanks for finding this.
> The change is actually in the assemble script in Bigtop, which should
> leave lib directories and the .txt files in the respective projects
> rather then moving them.
> I see you've already fixed the script. Thanks.
> I will upload the new artifact as soon as it built. There was a
> problem with disk space on ubuntu3.
>
> The question is should we reset the voting period.
> Since there was no change to the Hadoop tree I would rather not,
> unless people ask for it.
>
> Thanks,
> --Konstantin
>
> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>> <sh...@gmail.com> wrote:
>>> I created a release candidate for hadoop-0.22.0 available for review at:
>>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>
>> Pulled this RC into Bigtop and certified the following stack:
>> Hadoop 0.22
>> HBase 0.92
>> Zookeeper 3.4.0
>>
>> Found the following issues (which I suggest we fix and cut RC1):
>> 1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>> -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>> tar
>> [copy] Copying 48 files to
>> /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>>
>> BUILD FAILED
>> /tmp/hadoop-0.22.0/common/build.xml:1134:
>> /tmp/hadoop-0.22.0/common/lib not found.
>>
>> 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>> common, hdfs and mapred
>>
>> I will update Bigtop build scripts to take care of missing lib/
>> creation and also missing files.
>>
>> Once that's done we can cut a new RC when needed.
>>
>> Thanks,
>> Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Jakob Homan <jg...@gmail.com>.
+1 on the latest artifacts. Downloaded and verified md5 and signature.
2011/12/5 Scott Carey <sc...@richrelevance.com>:
>
>
> On 12/5/11 6:37 PM, "Konstantin Shvachko" <sh...@gmail.com> wrote:
>
>>Scott,
>>
>>I did not find anything in Apache policies about invalidating prior
>>+1's in case of changing the bits.
>
> I am assuming that the votes prior to the change are invalidated as a
> conclusion from what I understand to be an Apache release. The release
> vote is about the legal aspect of the release artifacts. Are all the
> contents are properly licensed and the package properly cryptographically
> signed? Therefore, if the contents and signatures change even in the most
> trivial way, then votes prior to the signature change cannot be taken to
> be votes on the release at all.
> http://www.apache.org/dev/release-publishing.html#signed
>
> I have no concern if you get 3 PMC +1's after the change and a majority of
> PMC +1's ('lazy majority').
>
>
>>This change was intended for those who want to build their own
>>binaries from the released source code. As I mentioned before there
>>was no change to the source tree.
>
> It is a requirement that an Apache release be buildable from source, not
> just contain source.
> http://www.apache.org/dev/release-publishing.html#valid
>
>>To accommodate your concern, I'd like to reiterate that there was a
>>trivial change in the directory structure of the release artifact. And
>>people who voted before Dec 3, 2011 5:45 PM PST should revoke their
>>votes if they think this change invalidated their +1's.
>
> I understand the change is trivial.
> I am only concerned if the release fails to get 3 PMC +1's after the
> change. Then I'd be a little uncertain of the official status of the vote
> (did at least 3 PMC members verify the signatures?). The PMC's duty in a
> release is to validate that the files released have valid contents _and_
> match their crypto signatures.
>
>>
>>Thanks,
>>--Konstantin
>>
>>
>>2011/12/5 Scott Carey <sc...@richrelevance.com>:
>>>
>>>
>>> On 12/3/11 10:52 PM, "Konstantin Boudnik" <co...@apache.org> wrote:
>>>
>>>>On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
>>>>> Roman,
>>>>>
>>>>> Thanks for finding this.
>>>>> The change is actually in the assemble script in Bigtop, which should
>>>>> leave lib directories and the .txt files in the respective projects
>>>>> rather then moving them.
>>>>> I see you've already fixed the script. Thanks.
>>>>> I will upload the new artifact as soon as it built. There was a
>>>>> problem with disk space on ubuntu3.
>>>>>
>>>>> The question is should we reset the voting period.
>>>>> Since there was no change to the Hadoop tree I would rather not,
>>>>> unless people ask for it.
>>>>
>>>>I agree with you: they artifacts are essentially the same, so there's no
>>>>need
>>>>to reset the vote IMO.
>>>>
>>>>Cos
>>>
>>> I am certain that anything that changes the bits in the release and thus
>>> the signatures invalidates prior +1's, by Apache policy.
>>> I do not know if any reset of the clock is required since there are no
>>> functional changes.
>>>
>>> Each person who voted +1 prior to the change can decide on their own
>>>what
>>> they need to do to vote +1 again -- that may be only a re-check of the
>>> signatures and validation that the change was trivial.
>>>
>>> In many other projects, even a simple correction of a typo creates a new
>>> release candidate.
>>>
>>>
>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> --Konstantin
>>>>>
>>>>> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org>
>>>>>wrote:
>>>>> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>>>>> > <sh...@gmail.com> wrote:
>>>>> >> I created a release candidate for hadoop-0.22.0 available for
>>>>>review
>>>>>at:
>>>>> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>>>> >
>>>>> > Pulled this RC into Bigtop and certified the following stack:
>>>>> > ═ Hadoop 0.22
>>>>> > ═ HBase 0.92
>>>>> > ═ Zookeeper 3.4.0
>>>>> >
>>>>> > Found the following issues (which I suggest we fix and cut RC1):
>>>>> > ═1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>>>>> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>>>>> > tar
>>>>> > ═ ═[copy] Copying 48 files to
>>>>> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>>>>> >
>>>>> > ═ ═BUILD FAILED
>>>>> > ═ ═/tmp/hadoop-0.22.0/common/build.xml:1134:
>>>>> > ═ ═/tmp/hadoop-0.22.0/common/lib not found.
>>>>> >
>>>>> > ═ 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>>>>> > common, hdfs and mapred
>>>>> >
>>>>> > I will update Bigtop build scripts to take care of missing lib/
>>>>> > creation and also missing files.
>>>>> >
>>>>> > Once that's done we can cut a new RC when needed.
>>>>> >
>>>>> > Thanks,
>>>>> > Roman.
>>>
>
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Yes, you are right, according to Hadoop bylaws.
http://hadoop.apache.org/bylaws.html#Decision+Making
--Konstantin
On Tue, Dec 6, 2011 at 11:55 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Tue, Dec 6, 2011 at 11:50 AM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
>> Ok. The vote will run till Saturday.
>
> That is, of course, fine.
>
> The question that I have is a different one: is my understanding of the
> process correct and THE ONLY thing that could potentially affect the
> outcome here is a lot of -1 PMC votes cast between now and Sat?
>
> Just trying to understand Hadoop release process a little better.
>
> Thanks,
> Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Dec 6, 2011 at 11:50 AM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Ok. The vote will run till Saturday.
That is, of course, fine.
The question that I have is a different one: is my understanding of the
process correct and THE ONLY thing that could potentially affect the
outcome here is a lot of -1 PMC votes cast between now and Sat?
Just trying to understand Hadoop release process a little better.
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Ok. The vote will run till Saturday.
Thanks,
--Konstantin
On Tue, Dec 6, 2011 at 10:51 AM, Owen O'Malley <ow...@hortonworks.com> wrote:
> On Tue, Dec 6, 2011 at 10:49 AM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
>> Owen,
>>
>> Could you please clarify. Do you mean the vote should ran 3 more days
>> or can be closed now?
>
> I'm saying that since you regenerated the tarball (md5's don't match),
> the vote should run 3 more days.
>
> -- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Owen O'Malley <ow...@hortonworks.com>.
On Tue, Dec 6, 2011 at 10:49 AM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> Owen,
>
> Could you please clarify. Do you mean the vote should ran 3 more days
> or can be closed now?
I'm saying that since you regenerated the tarball (md5's don't match),
the vote should run 3 more days.
-- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Owen,
Could you please clarify. Do you mean the vote should ran 3 more days
or can be closed now?
As mentioned earlier the file hadoop-0.22.0.tar.gz was regenerated
last Saturday due to a trivial change in the packaging script. The
Hadoop sources in it remained unchanged.
Please advise.
Thanks,
--Konstantin
On Tue, Dec 6, 2011 at 9:13 AM, Owen O'Malley <ow...@hortonworks.com> wrote:
> On Mon, Dec 5, 2011 at 10:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
>> I am sure things like that are controlled by projects' bylaws that's why
>> there's nothing specific at the apache level about that.
>
> The vote is required to be on a source tarball. Generated artifacts
> and subversion tags may be provided as a convenience, but are not the
> release. If the source tarball is the same, the vote holds. If there
> are any changes to the source tarball the vote should restart.
>
> -- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <Ko...@Boudnik.org>.
Thanks for the correction Owen - that makes sense.
On Tue, Dec 06, 2011 at 09:13AM, Owen O'Malley wrote:
> On Mon, Dec 5, 2011 at 10:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
> > I am sure things like that are controlled by projects' bylaws that's why
> > there's nothing specific at the apache level about that.
>
> The vote is required to be on a source tarball. Generated artifacts
> and subversion tags may be provided as a convenience, but are not the
> release. If the source tarball is the same, the vote holds. If there
> are any changes to the source tarball the vote should restart.
>
> -- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <co...@apache.org>.
Thanks for the correction Owen - that makes sense
On Tue, Dec 06, 2011 at 09:13AM, Owen O'Malley wrote:
> On Mon, Dec 5, 2011 at 10:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
> > I am sure things like that are controlled by projects' bylaws that's why
> > there's nothing specific at the apache level about that.
>
> The vote is required to be on a source tarball. Generated artifacts
> and subversion tags may be provided as a convenience, but are not the
> release. If the source tarball is the same, the vote holds. If there
> are any changes to the source tarball the vote should restart.
>
> -- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Dec 6, 2011 at 9:13 AM, Owen O'Malley <ow...@hortonworks.com> wrote:
> On Mon, Dec 5, 2011 at 10:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
>> I am sure things like that are controlled by projects' bylaws that's why
>> there's nothing specific at the apache level about that.
>
> The vote is required to be on a source tarball. Generated artifacts
> and subversion tags may be provided as a convenience, but are not the
> release. If the source tarball is the same, the vote holds. If there
> are any changes to the source tarball the vote should restart.
I believe there's been +3 PMC votes since the update (Doug, Jakob[recast]
and Konstantin) of the artifact and an extra +1 based on Bigtop testing.
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Owen O'Malley <ow...@hortonworks.com>.
On Mon, Dec 5, 2011 at 10:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
> I am sure things like that are controlled by projects' bylaws that's why
> there's nothing specific at the apache level about that.
The vote is required to be on a source tarball. Generated artifacts
and subversion tags may be provided as a convenience, but are not the
release. If the source tarball is the same, the vote holds. If there
are any changes to the source tarball the vote should restart.
-- Owen
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <co...@apache.org>.
I am sure things like that are controlled by projects' bylaws that's why
there's nothing specific at the apache level about that. As far as I remember
Hadoop's bylaws don't have any saying wrt a situation like this.
So, I believe the vote is good despite the change in the structure of an
archive which didn't lead to _any_ changes in the functionality.
Cos
On Mon, Dec 05, 2011 at 06:37PM, Konstantin Shvachko wrote:
> Scott,
>
> I did not find anything in Apache policies about invalidating prior
> +1's in case of changing the bits.
> This change was intended for those who want to build their own
> binaries from the released source code. As I mentioned before there
> was no change to the source tree.
> To accommodate your concern, I'd like to reiterate that there was a
> trivial change in the directory structure of the release artifact. And
> people who voted before Dec 3, 2011 5:45 PM PST should revoke their
> votes if they think this change invalidated their +1's.
>
> Thanks,
> --Konstantin
>
>
> 2011/12/5 Scott Carey <sc...@richrelevance.com>:
> >
> >
> > On 12/3/11 10:52 PM, "Konstantin Boudnik" <co...@apache.org> wrote:
> >
> >>On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
> >>> Roman,
> >>>
> >>> Thanks for finding this.
> >>> The change is actually in the assemble script in Bigtop, which should
> >>> leave lib directories and the .txt files in the respective projects
> >>> rather then moving them.
> >>> I see you've already fixed the script. Thanks.
> >>> I will upload the new artifact as soon as it built. There was a
> >>> problem with disk space on ubuntu3.
> >>>
> >>> The question is should we reset the voting period.
> >>> Since there was no change to the Hadoop tree I would rather not,
> >>> unless people ask for it.
> >>
> >>I agree with you: they artifacts are essentially the same, so there's no
> >>need
> >>to reset the vote IMO.
> >>
> >>Cos
> >
> > I am certain that anything that changes the bits in the release and thus
> > the signatures invalidates prior +1's, by Apache policy.
> > I do not know if any reset of the clock is required since there are no
> > functional changes.
> >
> > Each person who voted +1 prior to the change can decide on their own what
> > they need to do to vote +1 again -- that may be only a re-check of the
> > signatures and validation that the change was trivial.
> >
> > In many other projects, even a simple correction of a typo creates a new
> > release candidate.
> >
> >
> >
> >>
> >>>
> >>> Thanks,
> >>> --Konstantin
> >>>
> >>> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> >>> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
> >>> > <sh...@gmail.com> wrote:
> >>> >> I created a release candidate for hadoop-0.22.0 available for review
> >>>at:
> >>> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
> >>> >
> >>> > Pulled this RC into Bigtop and certified the following stack:
> >>> > ═ Hadoop 0.22
> >>> > ═ HBase 0.92
> >>> > ═ Zookeeper 3.4.0
> >>> >
> >>> > Found the following issues (which I suggest we fix and cut RC1):
> >>> > ═1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
> >>> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
> >>> > tar
> >>> > ═ ═[copy] Copying 48 files to
> >>> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
> >>> >
> >>> > ═ ═BUILD FAILED
> >>> > ═ ═/tmp/hadoop-0.22.0/common/build.xml:1134:
> >>> > ═ ═/tmp/hadoop-0.22.0/common/lib not found.
> >>> >
> >>> > ═ 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
> >>> > common, hdfs and mapred
> >>> >
> >>> > I will update Bigtop build scripts to take care of missing lib/
> >>> > creation and also missing files.
> >>> >
> >>> > Once that's done we can cut a new RC when needed.
> >>> >
> >>> > Thanks,
> >>> > Roman.
> >
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Scott Carey <sc...@richrelevance.com>.
On 12/5/11 6:37 PM, "Konstantin Shvachko" <sh...@gmail.com> wrote:
>Scott,
>
>I did not find anything in Apache policies about invalidating prior
>+1's in case of changing the bits.
I am assuming that the votes prior to the change are invalidated as a
conclusion from what I understand to be an Apache release. The release
vote is about the legal aspect of the release artifacts. Are all the
contents are properly licensed and the package properly cryptographically
signed? Therefore, if the contents and signatures change even in the most
trivial way, then votes prior to the signature change cannot be taken to
be votes on the release at all.
http://www.apache.org/dev/release-publishing.html#signed
I have no concern if you get 3 PMC +1's after the change and a majority of
PMC +1's ('lazy majority').
>This change was intended for those who want to build their own
>binaries from the released source code. As I mentioned before there
>was no change to the source tree.
It is a requirement that an Apache release be buildable from source, not
just contain source.
http://www.apache.org/dev/release-publishing.html#valid
>To accommodate your concern, I'd like to reiterate that there was a
>trivial change in the directory structure of the release artifact. And
>people who voted before Dec 3, 2011 5:45 PM PST should revoke their
>votes if they think this change invalidated their +1's.
I understand the change is trivial.
I am only concerned if the release fails to get 3 PMC +1's after the
change. Then I'd be a little uncertain of the official status of the vote
(did at least 3 PMC members verify the signatures?). The PMC's duty in a
release is to validate that the files released have valid contents _and_
match their crypto signatures.
>
>Thanks,
>--Konstantin
>
>
>2011/12/5 Scott Carey <sc...@richrelevance.com>:
>>
>>
>> On 12/3/11 10:52 PM, "Konstantin Boudnik" <co...@apache.org> wrote:
>>
>>>On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
>>>> Roman,
>>>>
>>>> Thanks for finding this.
>>>> The change is actually in the assemble script in Bigtop, which should
>>>> leave lib directories and the .txt files in the respective projects
>>>> rather then moving them.
>>>> I see you've already fixed the script. Thanks.
>>>> I will upload the new artifact as soon as it built. There was a
>>>> problem with disk space on ubuntu3.
>>>>
>>>> The question is should we reset the voting period.
>>>> Since there was no change to the Hadoop tree I would rather not,
>>>> unless people ask for it.
>>>
>>>I agree with you: they artifacts are essentially the same, so there's no
>>>need
>>>to reset the vote IMO.
>>>
>>>Cos
>>
>> I am certain that anything that changes the bits in the release and thus
>> the signatures invalidates prior +1's, by Apache policy.
>> I do not know if any reset of the clock is required since there are no
>> functional changes.
>>
>> Each person who voted +1 prior to the change can decide on their own
>>what
>> they need to do to vote +1 again -- that may be only a re-check of the
>> signatures and validation that the change was trivial.
>>
>> In many other projects, even a simple correction of a typo creates a new
>> release candidate.
>>
>>
>>
>>>
>>>>
>>>> Thanks,
>>>> --Konstantin
>>>>
>>>> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org>
>>>>wrote:
>>>> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>>>> > <sh...@gmail.com> wrote:
>>>> >> I created a release candidate for hadoop-0.22.0 available for
>>>>review
>>>>at:
>>>> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>>> >
>>>> > Pulled this RC into Bigtop and certified the following stack:
>>>> > �� Hadoop 0.22
>>>> > �� HBase 0.92
>>>> > �� Zookeeper 3.4.0
>>>> >
>>>> > Found the following issues (which I suggest we fix and cut RC1):
>>>> > ��1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>>>> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>>>> > tar
>>>> > �� ��[copy] Copying 48 files to
>>>> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>>>> >
>>>> > �� ��BUILD FAILED
>>>> > �� ��/tmp/hadoop-0.22.0/common/build.xml:1134:
>>>> > �� ��/tmp/hadoop-0.22.0/common/lib not found.
>>>> >
>>>> > �� 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>>>> > common, hdfs and mapred
>>>> >
>>>> > I will update Bigtop build scripts to take care of missing lib/
>>>> > creation and also missing files.
>>>> >
>>>> > Once that's done we can cut a new RC when needed.
>>>> >
>>>> > Thanks,
>>>> > Roman.
>>
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Scott,
I did not find anything in Apache policies about invalidating prior
+1's in case of changing the bits.
This change was intended for those who want to build their own
binaries from the released source code. As I mentioned before there
was no change to the source tree.
To accommodate your concern, I'd like to reiterate that there was a
trivial change in the directory structure of the release artifact. And
people who voted before Dec 3, 2011 5:45 PM PST should revoke their
votes if they think this change invalidated their +1's.
Thanks,
--Konstantin
2011/12/5 Scott Carey <sc...@richrelevance.com>:
>
>
> On 12/3/11 10:52 PM, "Konstantin Boudnik" <co...@apache.org> wrote:
>
>>On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
>>> Roman,
>>>
>>> Thanks for finding this.
>>> The change is actually in the assemble script in Bigtop, which should
>>> leave lib directories and the .txt files in the respective projects
>>> rather then moving them.
>>> I see you've already fixed the script. Thanks.
>>> I will upload the new artifact as soon as it built. There was a
>>> problem with disk space on ubuntu3.
>>>
>>> The question is should we reset the voting period.
>>> Since there was no change to the Hadoop tree I would rather not,
>>> unless people ask for it.
>>
>>I agree with you: they artifacts are essentially the same, so there's no
>>need
>>to reset the vote IMO.
>>
>>Cos
>
> I am certain that anything that changes the bits in the release and thus
> the signatures invalidates prior +1's, by Apache policy.
> I do not know if any reset of the clock is required since there are no
> functional changes.
>
> Each person who voted +1 prior to the change can decide on their own what
> they need to do to vote +1 again -- that may be only a re-check of the
> signatures and validation that the change was trivial.
>
> In many other projects, even a simple correction of a typo creates a new
> release candidate.
>
>
>
>>
>>>
>>> Thanks,
>>> --Konstantin
>>>
>>> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>>> > <sh...@gmail.com> wrote:
>>> >> I created a release candidate for hadoop-0.22.0 available for review
>>>at:
>>> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>>> >
>>> > Pulled this RC into Bigtop and certified the following stack:
>>> > ═ Hadoop 0.22
>>> > ═ HBase 0.92
>>> > ═ Zookeeper 3.4.0
>>> >
>>> > Found the following issues (which I suggest we fix and cut RC1):
>>> > ═1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>>> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>>> > tar
>>> > ═ ═[copy] Copying 48 files to
>>> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>>> >
>>> > ═ ═BUILD FAILED
>>> > ═ ═/tmp/hadoop-0.22.0/common/build.xml:1134:
>>> > ═ ═/tmp/hadoop-0.22.0/common/lib not found.
>>> >
>>> > ═ 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>>> > common, hdfs and mapred
>>> >
>>> > I will update Bigtop build scripts to take care of missing lib/
>>> > creation and also missing files.
>>> >
>>> > Once that's done we can cut a new RC when needed.
>>> >
>>> > Thanks,
>>> > Roman.
>
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Dec 3, 2011 at 10:52 PM, Konstantin Boudnik <co...@apache.org> wrote:
> I agree with you: they artifacts are essentially the same, so there's no need
> to reset the vote IMO.
Agreed. I compared the two tarballs and the changes don't warrant retesting
from the standpoint of functionality.
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Scott Carey <sc...@richrelevance.com>.
On 12/3/11 10:52 PM, "Konstantin Boudnik" <co...@apache.org> wrote:
>On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
>> Roman,
>>
>> Thanks for finding this.
>> The change is actually in the assemble script in Bigtop, which should
>> leave lib directories and the .txt files in the respective projects
>> rather then moving them.
>> I see you've already fixed the script. Thanks.
>> I will upload the new artifact as soon as it built. There was a
>> problem with disk space on ubuntu3.
>>
>> The question is should we reset the voting period.
>> Since there was no change to the Hadoop tree I would rather not,
>> unless people ask for it.
>
>I agree with you: they artifacts are essentially the same, so there's no
>need
>to reset the vote IMO.
>
>Cos
I am certain that anything that changes the bits in the release and thus
the signatures invalidates prior +1's, by Apache policy.
I do not know if any reset of the clock is required since there are no
functional changes.
Each person who voted +1 prior to the change can decide on their own what
they need to do to vote +1 again -- that may be only a re-check of the
signatures and validation that the change was trivial.
In many other projects, even a simple correction of a typo creates a new
release candidate.
>
>>
>> Thanks,
>> --Konstantin
>>
>> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
>> > <sh...@gmail.com> wrote:
>> >> I created a release candidate for hadoop-0.22.0 available for review
>>at:
>> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>> >
>> > Pulled this RC into Bigtop and certified the following stack:
>> > �� Hadoop 0.22
>> > �� HBase 0.92
>> > �� Zookeeper 3.4.0
>> >
>> > Found the following issues (which I suggest we fix and cut RC1):
>> > ��1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
>> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
>> > tar
>> > �� ��[copy] Copying 48 files to
>> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>> >
>> > �� ��BUILD FAILED
>> > �� ��/tmp/hadoop-0.22.0/common/build.xml:1134:
>> > �� ��/tmp/hadoop-0.22.0/common/lib not found.
>> >
>> > �� 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
>> > common, hdfs and mapred
>> >
>> > I will update Bigtop build scripts to take care of missing lib/
>> > creation and also missing files.
>> >
>> > Once that's done we can cut a new RC when needed.
>> >
>> > Thanks,
>> > Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Boudnik <co...@apache.org>.
On Sat, Dec 03, 2011 at 04:40PM, Konstantin Shvachko wrote:
> Roman,
>
> Thanks for finding this.
> The change is actually in the assemble script in Bigtop, which should
> leave lib directories and the .txt files in the respective projects
> rather then moving them.
> I see you've already fixed the script. Thanks.
> I will upload the new artifact as soon as it built. There was a
> problem with disk space on ubuntu3.
>
> The question is should we reset the voting period.
> Since there was no change to the Hadoop tree I would rather not,
> unless people ask for it.
I agree with you: they artifacts are essentially the same, so there's no need
to reset the vote IMO.
Cos
>
> Thanks,
> --Konstantin
>
> On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> > On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
> > <sh...@gmail.com> wrote:
> >> I created a release candidate for hadoop-0.22.0 available for review at:
> >> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
> >
> > Pulled this RC into Bigtop and certified the following stack:
> > ═ Hadoop 0.22
> > ═ HBase 0.92
> > ═ Zookeeper 3.4.0
> >
> > Found the following issues (which I suggest we fix and cut RC1):
> > ═1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
> > -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
> > tar
> > ═ ═[copy] Copying 48 files to
> > /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
> >
> > ═ ═BUILD FAILED
> > ═ ═/tmp/hadoop-0.22.0/common/build.xml:1134:
> > ═ ═/tmp/hadoop-0.22.0/common/lib not found.
> >
> > ═ 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
> > common, hdfs and mapred
> >
> > I will update Bigtop build scripts to take care of missing lib/
> > creation and also missing files.
> >
> > Once that's done we can cut a new RC when needed.
> >
> > Thanks,
> > Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Konstantin Shvachko <sh...@gmail.com>.
Roman,
Thanks for finding this.
The change is actually in the assemble script in Bigtop, which should
leave lib directories and the .txt files in the respective projects
rather then moving them.
I see you've already fixed the script. Thanks.
I will upload the new artifact as soon as it built. There was a
problem with disk space on ubuntu3.
The question is should we reset the voting period.
Since there was no change to the Hadoop tree I would rather not,
unless people ask for it.
Thanks,
--Konstantin
On Fri, Dec 2, 2011 at 7:34 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
> <sh...@gmail.com> wrote:
>> I created a release candidate for hadoop-0.22.0 available for review at:
>> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>
> Pulled this RC into Bigtop and certified the following stack:
> Hadoop 0.22
> HBase 0.92
> Zookeeper 3.4.0
>
> Found the following issues (which I suggest we fix and cut RC1):
> 1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
> -Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
> tar
> [copy] Copying 48 files to
> /tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
>
> BUILD FAILED
> /tmp/hadoop-0.22.0/common/build.xml:1134:
> /tmp/hadoop-0.22.0/common/lib not found.
>
> 2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
> common, hdfs and mapred
>
> I will update Bigtop build scripts to take care of missing lib/
> creation and also missing files.
>
> Once that's done we can cut a new RC when needed.
>
> Thanks,
> Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I created a release candidate for hadoop-0.22.0 available for review at:
> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
Pulled this RC into Bigtop and certified the following stack:
Hadoop 0.22
HBase 0.92
Zookeeper 3.4.0
Found the following issues (which I suggest we fix and cut RC1):
1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
-Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
tar
[copy] Copying 48 files to
/tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
BUILD FAILED
/tmp/hadoop-0.22.0/common/build.xml:1134:
/tmp/hadoop-0.22.0/common/lib not found.
2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
common, hdfs and mapred
I will update Bigtop build scripts to take care of missing lib/
creation and also missing files.
Once that's done we can cut a new RC when needed.
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Nov 29, 2011 at 2:47 AM, Konstantin Shvachko
<sh...@gmail.com> wrote:
> I created a release candidate for hadoop-0.22.0 available for review at:
> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
Pulled this RC into Bigtop and certified the following stack:
Hadoop 0.22
HBase 0.92
Zookeeper 3.4.0
Found the following issues (which I suggest we fix and cut RC1):
1. cd common ; ant -Dcompile.native=true -Dlibrecordio=true
-Dlibhdfs=1 -Dfusedfs=true -Dcompile.c++=true api-report bin-package
tar
[copy] Copying 48 files to
/tmp/hadoop-0.22.0/common/build/hadoop-common-0.22.0/lib
BUILD FAILED
/tmp/hadoop-0.22.0/common/build.xml:1134:
/tmp/hadoop-0.22.0/common/lib not found.
2. LICENSE.txt, NOTICE.txt and README.txt are missing from under
common, hdfs and mapred
I will update Bigtop build scripts to take care of missing lib/
creation and also missing files.
Once that's done we can cut a new RC when needed.
Thanks,
Roman.
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Doug Cutting <cu...@apache.org>.
+1 Checksums look good, RAT shows no significant license problems, tests
pass.
Doug
On 11/29/2011 02:47 AM, Konstantin Shvachko wrote:
> I created a release candidate for hadoop-0.22.0 available for review at:
> http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>
> The candidate incorporates more than 700 fixed jiras counting from the
> release of 0.21.0.
> The branch was cut on November 17 last year. During this year the
> branch was tested by different groups in the Hadoop community.
> Recently it has been extensively tested on a mid-size cluster.
> It contains important improvements, optimizations and bug fixes for
> HDFS and MapReduce compared to 0.21.0. Some features introduced in the
> latest offspring releases of hadoop-security are not yet ported to
> this branch, which is mostly related to security and MapReduce
> improvements. Compared to 0.23.0 this release will not contain
> Federation or Yarn.
> The hadoop-0.22.0 release build is fully automated thanks to the
> BigTop team. rc0 artifacts are available and reproducible on Hadoop
> Jenkins. The validity of rc0 has been also verified with BigTop.
> I think this could be a usable release and an important step forward
> in the Hadoop roadmap.
>
> Many people contributed to hadoop-0.22. Thank you everybody who
> developed code, built, tested, found and fixed bugs or supported this
> effort by any other means.
>
> Please try it out and vote.
>
> Thanks,
> --Konstantin
RE: [VOTE] Release hadoop-0.22.0-rc0
Posted by "Cheung, Po" <po...@ebay.com>.
+1
Thanks,
Po
-----Original Message-----
From: Konstantin Shvachko <sh...@gmail.com>
Date: Tue, Nov 29, 2011 at 2:47 AM
Subject: [VOTE] Release hadoop-0.22.0-rc0
To: general@hadoop.apache.org
I created a release candidate for hadoop-0.22.0 available for review at:
http://people.apache.org/~shv/hadoop-0.22.0-rc0/
The candidate incorporates more than 700 fixed jiras counting from the
release of 0.21.0.
The branch was cut on November 17 last year. During this year the
branch was tested by different groups in the Hadoop community.
Recently it has been extensively tested on a mid-size cluster.
It contains important improvements, optimizations and bug fixes for
HDFS and MapReduce compared to 0.21.0. Some features introduced in the
latest offspring releases of hadoop-security are not yet ported to
this branch, which is mostly related to security and MapReduce
improvements. Compared to 0.23.0 this release will not contain
Federation or Yarn.
The hadoop-0.22.0 release build is fully automated thanks to the
BigTop team. rc0 artifacts are available and reproducible on Hadoop
Jenkins. The validity of rc0 has been also verified with BigTop.
I think this could be a usable release and an important step forward
in the Hadoop roadmap.
Many people contributed to hadoop-0.22. Thank you everybody who
developed code, built, tested, found and fixed bugs or supported this
effort by any other means.
Please try it out and vote.
Thanks,
--Konstantin
Re: [VOTE] Release hadoop-0.22.0-rc0
Posted by Mi...@emc.com.
+1
Downloaded, installed on single node, (simple authentication), ran hadoop
mapred examples successfully.
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
On 11/29/11 2:47 AM, "Konstantin Shvachko" <sh...@gmail.com> wrote:
>I created a release candidate for hadoop-0.22.0 available for review at:
>http://people.apache.org/~shv/hadoop-0.22.0-rc0/
>
>The candidate incorporates more than 700 fixed jiras counting from the
>release of 0.21.0.
>The branch was cut on November 17 last year. During this year the
>branch was tested by different groups in the Hadoop community.
>Recently it has been extensively tested on a mid-size cluster.
>It contains important improvements, optimizations and bug fixes for
>HDFS and MapReduce compared to 0.21.0. Some features introduced in the
>latest offspring releases of hadoop-security are not yet ported to
>this branch, which is mostly related to security and MapReduce
>improvements. Compared to 0.23.0 this release will not contain
>Federation or Yarn.
>The hadoop-0.22.0 release build is fully automated thanks to the
>BigTop team. rc0 artifacts are available and reproducible on Hadoop
>Jenkins. The validity of rc0 has been also verified with BigTop.
>I think this could be a usable release and an important step forward
>in the Hadoop roadmap.
>
>Many people contributed to hadoop-0.22. Thank you everybody who
>developed code, built, tested, found and fixed bugs or supported this
>effort by any other means.
>
>Please try it out and vote.
>
>Thanks,
>--Konstantin
>