You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Junping Du <jd...@hortonworks.com> on 2017/03/17 09:18:15 UTC
[VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Fwd: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
There's an RC of Hadoop 2.8.0 out for people to build and test against. This is the last chance for people to find problems and complain before it gets out the door
Spark happily builds against it. my tests all work. For anyone wanting to do the build, the maven arguments are: -Phive,yarn,hadoop-2.7 -Dhadoop.version=2.8.0 -Psnapshots-and-staging
One thing I would be curious about is how well things work on Windows, especially how informative it is if things like winutils.exe can't be found. Previous versions: stack trace on startup, even if not relevant, and meaningless text on use. With HADOOP-10775, stack traces should be postponed until point of use, and error text including a ref to : https://wiki.apache.org/hadoop/WindowsProblems <https://wiki.apache.org/hadoop/WindowsProblems>
I would really like to know how well that works for people.
Here's the win 64 native binaries to go with the ASF artifacts; built off the same commit
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3 <https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3>
> Begin forwarded message:
>
> From: Junping Du <jd...@hortonworks.com>
> Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> Date: 17 March 2017 at 09:18:15 GMT
> To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>, "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
>
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Brahma Reddy Battula <br...@huawei.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs
Regards
Brahma Reddy Battula
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Brahma Reddy Battula <br...@huawei.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs
Regards
Brahma Reddy Battula
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Marton Elek <me...@hortonworks.com>.
+1 (non-binding)
Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers)
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS
Looks good.
Marton
> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
>
> +1 (non-binding). Thanks Junping for the effort.
>
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
>
> Thanks
> Sunil
>
>
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Marton Elek <me...@hortonworks.com>.
+1 (non-binding)
Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers)
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS
Looks good.
Marton
> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
>
> +1 (non-binding). Thanks Junping for the effort.
>
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
>
> Thanks
> Sunil
>
>
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Marton Elek <me...@hortonworks.com>.
+1 (non-binding)
Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers)
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS
Looks good.
Marton
> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
>
> +1 (non-binding). Thanks Junping for the effort.
>
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
>
> Thanks
> Sunil
>
>
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Sunil Govind <su...@gmail.com>.
+1 (non-binding). Thanks Junping for the effort.
I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine
Thanks
Sunil
On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by surendra lilhore <su...@huawei.com>.
Hi,
I have used release package and verified basic operations for HDFS.
I raised one blocker issue for this release.
HADOOP-14207 : "dfsadmin -refreshCallQueue" command is failing with DecayRpcScheduler
Thanks,
Surendra
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 14:48
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Brahma Reddy Battula <br...@huawei.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs
Regards
Brahma Reddy Battula
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Miklos Szegedi <mi...@cloudera.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
I verified the following:
1. Deployed on a 3 node cluster with 2 node managers.
2. Configured linux container executor
3. Configured fair scheduler
4. Ran Pi job and verified the results
5. Ran multiple Yarn applications and verified the results
Thank you,
Miklos Szegedi
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Payne <er...@yahoo.com.INVALID>.
+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI
________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process. So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot. Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process. So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot. Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process. So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot. Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Yufei Gu <fl...@gmail.com>.
Thank Junping for working on this.
I verified the following:
1. Verified the md5 of binary tar ball.
2. Deployed on a 4 node cluster with 3 node managers.
3. Configured fair scheduler
4. Ran Pi job and verified the results.
5. Ran SLS for fair scheduler and capacity scheduler.
All good except SLS, it doesn't work. I guess this is an known issue.
Best,
Yufei
On Tue, Mar 21, 2017 at 10:12 AM, Andrew Wang <an...@cloudera.com>
wrote:
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
>
> It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
> extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
> content:
>
> -> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
> {print SUM}'
> 1651891481
>
>
> On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
> > Thanks Junping for taking on this huge effort!
> >
> > I found one tiny nit: the md5 files are not in conventional format
> > That is to say,
> > $ cat hadoop-2.8.0.tar.gz.md5
> >
> > $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> > c728a090b68d009070085367695ed507 /build/source/target/
> > artifacts/hadoop-2.8.0.tar.gz
> >
> > But a typical md5 file would have been:
> > c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
> >
> > I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> > binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> > But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> > comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> > amount of changes made into Hadoop 2.8 makes such difference in size.
> >
> > Regards,
> > Wei-Chiu Chuang
> >
> > > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org>
> wrote:
> > >
> > > Thanks Junping!
> > >
> > > +1 (binding)
> > >
> > > * Verified signatures and checksums
> > > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > > * Deployed a single node cluster and ran some Hive on Tez queries
> > successfully
> > > * The document looks good.
> > >
> > > I found a trivial issue in the doc. It does not block the release.
> > > https://issues.apache.org/jira/browse/HADOOP-14208
> > >
> > > Regards,
> > > Akira
> > >
> > >
> > > On 2017/03/17 18:18, Junping Du wrote:
> > >> Hi all,
> > >> With fix of HDFS-11431 get in, I've created a new release
> candidate
> > (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >> This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> > >>
> > >> More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >> New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> > >>
> > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >> The maven artifacts are available via repository.apache.org at:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1057
> > >>
> > >> Please try the release and vote; the vote will run for the usual
> 5
> > days, ending on 03/22/2017 PDT time.
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process. So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot. Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Andrew Wang <an...@cloudera.com>.
I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.
It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:
-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481
On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507 /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Andrew Wang <an...@cloudera.com>.
I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.
It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:
-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481
On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507 /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Andrew Wang <an...@cloudera.com>.
I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.
It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:
-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481
On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507 /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Andrew Wang <an...@cloudera.com>.
I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.
It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:
-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481
On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:
> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507 /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Thanks Junping for taking on this huge effort!
I found one tiny nit: the md5 files are not in conventional format
That is to say,
$ cat hadoop-2.8.0.tar.gz.md5
$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507 /build/source/target/artifacts/hadoop-2.8.0.tar.gz
But a typical md5 file would have been:
c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.
Regards,
Wei-Chiu Chuang
> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
>
> Thanks Junping!
>
> +1 (binding)
>
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
>
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
>
> Regards,
> Akira
>
>
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>
>> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Thanks Junping for taking on this huge effort!
I found one tiny nit: the md5 files are not in conventional format
That is to say,
$ cat hadoop-2.8.0.tar.gz.md5
$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507 /build/source/target/artifacts/hadoop-2.8.0.tar.gz
But a typical md5 file would have been:
c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.
Regards,
Wei-Chiu Chuang
> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
>
> Thanks Junping!
>
> +1 (binding)
>
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
>
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
>
> Regards,
> Akira
>
>
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>
>> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Thanks Junping for taking on this huge effort!
I found one tiny nit: the md5 files are not in conventional format
That is to say,
$ cat hadoop-2.8.0.tar.gz.md5
$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507 /build/source/target/artifacts/hadoop-2.8.0.tar.gz
But a typical md5 file would have been:
c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.
Regards,
Wei-Chiu Chuang
> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
>
> Thanks Junping!
>
> +1 (binding)
>
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
>
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
>
> Regards,
> Akira
>
>
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>
>> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Thanks Junping for taking on this huge effort!
I found one tiny nit: the md5 files are not in conventional format
That is to say,
$ cat hadoop-2.8.0.tar.gz.md5
$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507 /build/source/target/artifacts/hadoop-2.8.0.tar.gz
But a typical md5 file would have been:
c728a090b68d009070085367695ed507 hadoop-2.8.0.tar.gz
I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.
Regards,
Wei-Chiu Chuang
> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
>
> Thanks Junping!
>
> +1 (binding)
>
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
>
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
>
> Regards,
> Akira
>
>
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>
>> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Akira Ajisaka <aa...@apache.org>.
Thanks Junping!
+1 (binding)
* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries
successfully
* The document looks good.
I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208
Regards,
Akira
On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Gergo Pasztor <pa...@cloudera.com>.
+1 (non-binding)
- verified signatures
- run hadoop mapreduce examples
- use hdfs through shell
Best regards,
Gergő Pásztor
On Tue, Mar 21, 2017 at 5:17 PM, Masatake Iwasaki <
iwasakims@oss.nttdata.co.jp> wrote:
> Thanks for putting this up, Junping.
>
> +1(non-binding)
>
> * verified signature and mds.
> * built from source on centos 7 and openjdk 8 with native profile.
> * built docs and skimmed the contents.
> * deployed 3 nodes ha cluster and run some sample jobs with caller context
> enabled.
>
> Masatake Iwasaki
>
>
> On 3/17/17 18:18, Junping Du wrote:
>
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junpin
>> g_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for putting this up, Junping.
+1(non-binding)
* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller
context enabled.
Masatake Iwasaki
On 3/17/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for putting this up, Junping.
+1(non-binding)
* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller
context enabled.
Masatake Iwasaki
On 3/17/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Akira Ajisaka <aa...@apache.org>.
Thanks Junping!
+1 (binding)
* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries
successfully
* The document looks good.
I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208
Regards,
Akira
On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Mingliang Liu <li...@gmail.com>.
Thanks Junping for doing this.
+1 (non-binding)
0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs
L
> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Akira Ajisaka <aa...@apache.org>.
Thanks Junping!
+1 (binding)
* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries
successfully
* The document looks good.
I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208
Regards,
Akira
On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for doing this.
*+1 (Binding),*
Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.
Thanks,
Wangda
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for doing this.
*+1 (Binding),*
Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.
Thanks,
Wangda
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Naganarasimha Garla <na...@apache.org>.
Thanks Junping for putting this up,.
+1 (non-binding)
* downloaded verified signature and md5.
* Built from source & deployed pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.
Regards,
+ Naga
On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>> - Verified checksums and signatures of the tarballs
> > >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>> - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>> - Cloud connectors:
> > >>> - s3a: integration tests, basic fs commands
> > >>> - adl: live unit tests, basic fs commands. See notes below.
> > >>> - Deployed a pseudo cluster, passed the following sanity tests in
> > >>> both insecure and SSL mode:
> > >>> - HDFS: basic dfs, distcp, ACL commands
> > >>> - KMS and HttpFS: basic tests
> > >>> - MapReduce wordcount
> > >>> - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>> - HDFS-11132. Allow AccessControlException in contract tests when
> > >>> getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>> Dusane
> > >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>> runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>> With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>> This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>> More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>> New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>> The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>> Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Naganarasimha Garla <na...@apache.org>.
Thanks Junping for putting this up,.
+1 (non-binding)
* downloaded verified signature and md5.
* Built from source & deployed pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.
Regards,
+ Naga
On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>> - Verified checksums and signatures of the tarballs
> > >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>> - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>> - Cloud connectors:
> > >>> - s3a: integration tests, basic fs commands
> > >>> - adl: live unit tests, basic fs commands. See notes below.
> > >>> - Deployed a pseudo cluster, passed the following sanity tests in
> > >>> both insecure and SSL mode:
> > >>> - HDFS: basic dfs, distcp, ACL commands
> > >>> - KMS and HttpFS: basic tests
> > >>> - MapReduce wordcount
> > >>> - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>> - HDFS-11132. Allow AccessControlException in contract tests when
> > >>> getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>> Dusane
> > >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>> runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>> With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>> This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>> More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>> New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>> The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>> Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Naganarasimha Garla <na...@apache.org>.
Thanks Junping for putting this up,.
+1 (non-binding)
* downloaded verified signature and md5.
* Built from source & deployed pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.
Regards,
+ Naga
On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>> - Verified checksums and signatures of the tarballs
> > >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>> - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>> - Cloud connectors:
> > >>> - s3a: integration tests, basic fs commands
> > >>> - adl: live unit tests, basic fs commands. See notes below.
> > >>> - Deployed a pseudo cluster, passed the following sanity tests in
> > >>> both insecure and SSL mode:
> > >>> - HDFS: basic dfs, distcp, ACL commands
> > >>> - KMS and HttpFS: basic tests
> > >>> - MapReduce wordcount
> > >>> - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>> - HDFS-11132. Allow AccessControlException in contract tests when
> > >>> getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>> Dusane
> > >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>> runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>> With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>> This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>> More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>> New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>> The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>> Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Naganarasimha Garla <na...@apache.org>.
Thanks Junping for putting this up,.
+1 (non-binding)
* downloaded verified signature and md5.
* Built from source & deployed pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.
Regards,
+ Naga
On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:
> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>> - Verified checksums and signatures of the tarballs
> > >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>> - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>> - Cloud connectors:
> > >>> - s3a: integration tests, basic fs commands
> > >>> - adl: live unit tests, basic fs commands. See notes below.
> > >>> - Deployed a pseudo cluster, passed the following sanity tests in
> > >>> both insecure and SSL mode:
> > >>> - HDFS: basic dfs, distcp, ACL commands
> > >>> - KMS and HttpFS: basic tests
> > >>> - MapReduce wordcount
> > >>> - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>> - HDFS-11132. Allow AccessControlException in contract tests when
> > >>> getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>> Dusane
> > >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>> runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>> With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>> This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>> More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>> New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>> The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>> Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
Thanks Junping for getting this out.
+1 (non-binding)
* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds
Rakesh
On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>> - Verified checksums and signatures of the tarballs
> >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>> - Cloud connectors:
> >>> - s3a: integration tests, basic fs commands
> >>> - adl: live unit tests, basic fs commands. See notes below.
> >>> - Deployed a pseudo cluster, passed the following sanity tests in
> >>> both insecure and SSL mode:
> >>> - HDFS: basic dfs, distcp, ACL commands
> >>> - KMS and HttpFS: basic tests
> >>> - MapReduce wordcount
> >>> - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>> - HDFS-11132. Allow AccessControlException in contract tests when
> >>> getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>> Dusane
> >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>> runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>> <name>fs.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>> With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>> This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>> More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>> New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>> The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>> Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
Thanks Junping for getting this out.
+1 (non-binding)
* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds
Rakesh
On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>> - Verified checksums and signatures of the tarballs
> >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>> - Cloud connectors:
> >>> - s3a: integration tests, basic fs commands
> >>> - adl: live unit tests, basic fs commands. See notes below.
> >>> - Deployed a pseudo cluster, passed the following sanity tests in
> >>> both insecure and SSL mode:
> >>> - HDFS: basic dfs, distcp, ACL commands
> >>> - KMS and HttpFS: basic tests
> >>> - MapReduce wordcount
> >>> - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>> - HDFS-11132. Allow AccessControlException in contract tests when
> >>> getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>> Dusane
> >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>> runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>> <name>fs.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>> With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>> This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>> More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>> New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>> The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>> Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
Thanks Junping for getting this out.
+1 (non-binding)
* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds
Rakesh
On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>> - Verified checksums and signatures of the tarballs
> >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>> - Cloud connectors:
> >>> - s3a: integration tests, basic fs commands
> >>> - adl: live unit tests, basic fs commands. See notes below.
> >>> - Deployed a pseudo cluster, passed the following sanity tests in
> >>> both insecure and SSL mode:
> >>> - HDFS: basic dfs, distcp, ACL commands
> >>> - KMS and HttpFS: basic tests
> >>> - MapReduce wordcount
> >>> - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>> - HDFS-11132. Allow AccessControlException in contract tests when
> >>> getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>> Dusane
> >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>> runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>> <name>fs.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>> With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>> This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>> More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>> New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>> The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>> Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Rakesh Radhakrishnan <ra...@apache.org>.
Thanks Junping for getting this out.
+1 (non-binding)
* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds
Rakesh
On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>> - Verified checksums and signatures of the tarballs
> >>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>> - Cloud connectors:
> >>> - s3a: integration tests, basic fs commands
> >>> - adl: live unit tests, basic fs commands. See notes below.
> >>> - Deployed a pseudo cluster, passed the following sanity tests in
> >>> both insecure and SSL mode:
> >>> - HDFS: basic dfs, distcp, ACL commands
> >>> - KMS and HttpFS: basic tests
> >>> - MapReduce wordcount
> >>> - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>> - HDFS-11132. Allow AccessControlException in contract tests when
> >>> getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>> Dusane
> >>> - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>> runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>> <name>fs.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>> <name>fs.AbstractFileSystem.adl.impl</name>
> >>>> <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>> With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>> This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>> More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>> New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>> The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>> Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jian He <jh...@hortonworks.com>.
+1 (binding)
- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.
Jian
> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
>
> +1 (non-binding)
>
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
>
>
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
>
>> Thanks for all the effort Junping!
>>
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>>
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>>
>> Thanks
>> Ravi
>>
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Thanks all for response with verification work and vote!
>>>
>>>
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>>
>>>
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>>
>>>
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>>
>>> +1 (non-binding)
>>>
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>>
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>>
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>>
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>> +1. Thanks for the great effort, Junping!
>>>
>>>
>>> - Verified checksums and signatures of the tarballs
>>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>> - Cloud connectors:
>>> - s3a: integration tests, basic fs commands
>>> - adl: live unit tests, basic fs commands. See notes below.
>>> - Deployed a pseudo cluster, passed the following sanity tests in
>>> both insecure and SSL mode:
>>> - HDFS: basic dfs, distcp, ACL commands
>>> - KMS and HttpFS: basic tests
>>> - MapReduce wordcount
>>> - balancer start/stop
>>>
>>>
>>> Needs the following JIRAs to pass all ADL tests:
>>>
>>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>> - HDFS-11132. Allow AccessControlException in contract tests when
>>> getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>> Dusane
>>> - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>> runtime error. (John Zhuge via lei)
>>>
>>>
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>>
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>>
>>>> <property>
>>>> <name>fs.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>fs.AbstractFileSystem.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>>
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>>
>>>> Failed tests:
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>>
>>>> Tests in error:
>>>>
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>>
>>>>
>>>> Stay tuned...
>>>>
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>>
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>>
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>>
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>>
>>>>> I am working on a patch.
>>>>>
>>>>>
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>>
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>>
>>>>>> This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>>
>>>>>> More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>>
>>>>>> New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>>
>>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>>
>>>>>> The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>>
>>>>>> Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Junping
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> John
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jian He <jh...@hortonworks.com>.
+1 (binding)
- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.
Jian
> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
>
> +1 (non-binding)
>
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
>
>
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
>
>> Thanks for all the effort Junping!
>>
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>>
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>>
>> Thanks
>> Ravi
>>
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Thanks all for response with verification work and vote!
>>>
>>>
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>>
>>>
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>>
>>>
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>>
>>> +1 (non-binding)
>>>
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>>
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>>
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>>
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>> +1. Thanks for the great effort, Junping!
>>>
>>>
>>> - Verified checksums and signatures of the tarballs
>>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>> - Cloud connectors:
>>> - s3a: integration tests, basic fs commands
>>> - adl: live unit tests, basic fs commands. See notes below.
>>> - Deployed a pseudo cluster, passed the following sanity tests in
>>> both insecure and SSL mode:
>>> - HDFS: basic dfs, distcp, ACL commands
>>> - KMS and HttpFS: basic tests
>>> - MapReduce wordcount
>>> - balancer start/stop
>>>
>>>
>>> Needs the following JIRAs to pass all ADL tests:
>>>
>>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>> - HDFS-11132. Allow AccessControlException in contract tests when
>>> getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>> Dusane
>>> - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>> runtime error. (John Zhuge via lei)
>>>
>>>
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>>
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>>
>>>> <property>
>>>> <name>fs.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>fs.AbstractFileSystem.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>>
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>>
>>>> Failed tests:
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>>
>>>> Tests in error:
>>>>
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>>
>>>>
>>>> Stay tuned...
>>>>
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>>
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>>
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>>
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>>
>>>>> I am working on a patch.
>>>>>
>>>>>
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>>
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>>
>>>>>> This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>>
>>>>>> More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>>
>>>>>> New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>>
>>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>>
>>>>>> The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>>
>>>>>> Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Junping
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> John
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jian He <jh...@hortonworks.com>.
+1 (binding)
- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.
Jian
> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
>
> +1 (non-binding)
>
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
>
>
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
>
>> Thanks for all the effort Junping!
>>
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>>
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>>
>> Thanks
>> Ravi
>>
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Thanks all for response with verification work and vote!
>>>
>>>
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>>
>>>
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>>
>>>
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>>
>>> +1 (non-binding)
>>>
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>>
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>>
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>>
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>> +1. Thanks for the great effort, Junping!
>>>
>>>
>>> - Verified checksums and signatures of the tarballs
>>> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>> - Cloud connectors:
>>> - s3a: integration tests, basic fs commands
>>> - adl: live unit tests, basic fs commands. See notes below.
>>> - Deployed a pseudo cluster, passed the following sanity tests in
>>> both insecure and SSL mode:
>>> - HDFS: basic dfs, distcp, ACL commands
>>> - KMS and HttpFS: basic tests
>>> - MapReduce wordcount
>>> - balancer start/stop
>>>
>>>
>>> Needs the following JIRAs to pass all ADL tests:
>>>
>>> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>> - HDFS-11132. Allow AccessControlException in contract tests when
>>> getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>> Dusane
>>> - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>> runtime error. (John Zhuge via lei)
>>>
>>>
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>>
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>>
>>>> <property>
>>>> <name>fs.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>fs.AbstractFileSystem.adl.impl</name>
>>>> <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>>
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>>
>>>> Failed tests:
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>>
>>>> Tests in error:
>>>>
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>>
>>>>
>>>> Stay tuned...
>>>>
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>>
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>>
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>>
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>>
>>>>> I am working on a patch.
>>>>>
>>>>>
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>>
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>>
>>>>>> This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>>
>>>>>> More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>>
>>>>>> New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>>
>>>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>>
>>>>>> The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>>
>>>>>> Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Junping
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> John
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by larry mccay <la...@gmail.com>.
+1 (non-binding)
- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox
On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> > - Verified checksums and signatures of the tarballs
> > - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> > - Cloud connectors:
> > - s3a: integration tests, basic fs commands
> > - adl: live unit tests, basic fs commands. See notes below.
> > - Deployed a pseudo cluster, passed the following sanity tests in
> > both insecure and SSL mode:
> > - HDFS: basic dfs, distcp, ACL commands
> > - KMS and HttpFS: basic tests
> > - MapReduce wordcount
> > - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> > - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> > - HDFS-11132. Allow AccessControlException in contract tests when
> > getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> > Dusane
> > - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> > runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >> With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >> This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >> More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >> New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >> The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >> Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by larry mccay <la...@gmail.com>.
+1 (non-binding)
- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox
On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> > - Verified checksums and signatures of the tarballs
> > - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> > - Cloud connectors:
> > - s3a: integration tests, basic fs commands
> > - adl: live unit tests, basic fs commands. See notes below.
> > - Deployed a pseudo cluster, passed the following sanity tests in
> > both insecure and SSL mode:
> > - HDFS: basic dfs, distcp, ACL commands
> > - KMS and HttpFS: basic tests
> > - MapReduce wordcount
> > - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> > - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> > - HDFS-11132. Allow AccessControlException in contract tests when
> > getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> > Dusane
> > - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> > runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >> With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >> This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >> More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >> New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >> The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >> Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by larry mccay <la...@gmail.com>.
+1 (non-binding)
- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox
On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> > - Verified checksums and signatures of the tarballs
> > - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> > - Cloud connectors:
> > - s3a: integration tests, basic fs commands
> > - adl: live unit tests, basic fs commands. See notes below.
> > - Deployed a pseudo cluster, passed the following sanity tests in
> > both insecure and SSL mode:
> > - HDFS: basic dfs, distcp, ACL commands
> > - KMS and HttpFS: basic tests
> > - MapReduce wordcount
> > - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> > - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> > - HDFS-11132. Allow AccessControlException in contract tests when
> > getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> > Dusane
> > - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> > runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >> With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >> This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >> More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >> New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >> The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >> Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by larry mccay <la...@gmail.com>.
+1 (non-binding)
- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox
On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> > - Verified checksums and signatures of the tarballs
> > - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> > - Cloud connectors:
> > - s3a: integration tests, basic fs commands
> > - adl: live unit tests, basic fs commands. See notes below.
> > - Deployed a pseudo cluster, passed the following sanity tests in
> > both insecure and SSL mode:
> > - HDFS: basic dfs, distcp, ACL commands
> > - KMS and HttpFS: basic tests
> > - MapReduce wordcount
> > - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> > - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> > - HDFS-11132. Allow AccessControlException in contract tests when
> > getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> > Dusane
> > - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> > runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >> With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >> This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >> More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >> New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >> The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >> Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Ravi Prakash <ra...@gmail.com>.
Thanks for all the effort Junping!
+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.
I agree none of the issues reported so far are blockers. Looking forward to
another great release.
Thanks
Ravi
On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
> - Verified checksums and signatures of the tarballs
> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> - Cloud connectors:
> - s3a: integration tests, basic fs commands
> - adl: live unit tests, basic fs commands. See notes below.
> - Deployed a pseudo cluster, passed the following sanity tests in
> both insecure and SSL mode:
> - HDFS: basic dfs, distcp, ACL commands
> - KMS and HttpFS: basic tests
> - MapReduce wordcount
> - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
> - HDFS-11132. Allow AccessControlException in contract tests when
> getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
> Dusane
> - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
> runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >> With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >> This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >> More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >> New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >> The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >> Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Ravi Prakash <ra...@gmail.com>.
Thanks for all the effort Junping!
+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.
I agree none of the issues reported so far are blockers. Looking forward to
another great release.
Thanks
Ravi
On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
> - Verified checksums and signatures of the tarballs
> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> - Cloud connectors:
> - s3a: integration tests, basic fs commands
> - adl: live unit tests, basic fs commands. See notes below.
> - Deployed a pseudo cluster, passed the following sanity tests in
> both insecure and SSL mode:
> - HDFS: basic dfs, distcp, ACL commands
> - KMS and HttpFS: basic tests
> - MapReduce wordcount
> - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
> - HDFS-11132. Allow AccessControlException in contract tests when
> getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
> Dusane
> - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
> runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >> With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >> This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >> More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >> New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >> The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >> Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Ravi Prakash <ra...@gmail.com>.
Thanks for all the effort Junping!
+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.
I agree none of the issues reported so far are blockers. Looking forward to
another great release.
Thanks
Ravi
On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
> - Verified checksums and signatures of the tarballs
> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> - Cloud connectors:
> - s3a: integration tests, basic fs commands
> - adl: live unit tests, basic fs commands. See notes below.
> - Deployed a pseudo cluster, passed the following sanity tests in
> both insecure and SSL mode:
> - HDFS: basic dfs, distcp, ACL commands
> - KMS and HttpFS: basic tests
> - MapReduce wordcount
> - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
> - HDFS-11132. Allow AccessControlException in contract tests when
> getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
> Dusane
> - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
> runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >> With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >> This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >> More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >> New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >> The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >> Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Ravi Prakash <ra...@gmail.com>.
Thanks for all the effort Junping!
+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.
I agree none of the issues reported so far are blockers. Looking forward to
another great release.
Thanks
Ravi
On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
> - Verified checksums and signatures of the tarballs
> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> - Cloud connectors:
> - s3a: integration tests, basic fs commands
> - adl: live unit tests, basic fs commands. See notes below.
> - Deployed a pseudo cluster, passed the following sanity tests in
> both insecure and SSL mode:
> - HDFS: basic dfs, distcp, ACL commands
> - KMS and HttpFS: basic tests
> - MapReduce wordcount
> - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
> - HDFS-11132. Allow AccessControlException in contract tests when
> getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
> Dusane
> - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
> runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >> With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >> This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >> More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >> New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >> The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >> Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks all for response with verification work and vote!
Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.
We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!
Thanks,
Junping
________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks all for response with verification work and vote!
Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.
We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!
Thanks,
Junping
________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks all for response with verification work and vote!
Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.
We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!
Thanks,
Junping
________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks all for response with verification work and vote!
Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.
We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!
Thanks,
Junping
________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by zhenke luo <zh...@gmail.com>.
Hello:
I found an issue with HADOOP-10027 that :
SnappyCompressor can’t load native lib properly.
Test TestHadoopnative is failing in hadoop-apache2 version 0.10
stacktrace:
Caused by: java.lang.RuntimeException: native snappy library not available: SnappyCompressor has not been loaded.
at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:72)
at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:195)
at com.facebook.presto.hadoop.HadoopNative.loadAllCodecs(HadoopNative.java:71)
at com.facebook.presto.hadoop.HadoopNative.requireHadoopNative(HadoopNative.java:52)
... 24 more
After look into the code found that in SnappyCompressor.java
static {
if (NativeCodeLoader.isNativeCodeLoaded() &&
NativeCodeLoader.buildSupportsSnappy()) {
try {
initIDs();
nativeSnappyLoaded = true;
} catch (Throwable t) {
LOG.error("failed to load SnappyCompressor", t);
}
}
}
the native method initIDs will complain: no such field clazz.
> On Mar 21, 2017, at 4:32 PM, Poepping, Thomas <po...@amazon.com> wrote:
>
>
>
> On 3/21/17, 3:17 PM, "Kuhu Shukla" <ks...@yahoo-inc.com.INVALID> wrote:
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
> - Verified checksums and signatures of the tarballs
> - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> - Cloud connectors:
> - s3a: integration tests, basic fs commands
> - adl: live unit tests, basic fs commands. See notes below.
> - Deployed a pseudo cluster, passed the following sanity tests in
> both insecure and SSL mode:
> - HDFS: basic dfs, distcp, ACL commands
> - KMS and HttpFS: basic tests
> - MapReduce wordcount
> - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
> - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
> - HDFS-11132. Allow AccessControlException in contract tests when
> getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
> Dusane
> - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
> runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>> <name>fs.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>> <name>fs.AbstractFileSystem.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>>
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>>
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>> » AccessControl
>>
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>> » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Thank you for reporting the issue, John! Does this issue only affect ADL
>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>> existing
>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>> not a
>>> regression and just a new feature get broken.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>> ------------------------------
>>> *From:* John Zhuge <jz...@cloudera.com>
>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>> *To:* Junping Du
>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>> FileSystem for scheme: adl".
>>>
>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>> earlier.
>>> HADOOP-12666 should not be backported, but some changes are needed:
>>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>>
>>> I am working on a patch.
>>>
>>>
>>> John Zhuge
>>> Software Engineer, Cloudera
>>>
>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>>>
>>>> Hi all,
>>>> With fix of HDFS-11431 get in, I've created a new release candidate
>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>
>>>> This is the next minor release to follow up 2.7.0 which has been
>>>> released for more than 1 year. It comprises 2,900+ fixes, improvements,
>> and
>>>> new features. Most of these commits are released for the first time in
>>>> branch-2.
>>>>
>>>> More information about the 2.8.0 release plan can be found here:
>>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>>>
>>>> New RC is available at: http://home.apache.org/~junpin
>>>> g_du/hadoop-2.8.0-RC3
>>>>
>>>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>
>>>> The maven artifacts are available via repository.apache.org at:
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>>>
>>>> Please try the release and vote; the vote will run for the usual 5
>>>> days, ending on 03/22/2017 PDT time.
>
>>>>
>>>> Thanks,
>>>>
>>>> Junping
>>>>
>>>
>>>
>>
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Kuhu Shukla <ks...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Kuhu Shukla <ks...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Kuhu Shukla <ks...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Kuhu Shukla <ks...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
Thank you Junping and everyone else who worked on getting this release out.
Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Badger <eb...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Badger <eb...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Badger <eb...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Badger <eb...@yahoo-inc.com.INVALID>.
+1 (non-binding)
- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs
Thanks,
Eric
On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jo...@gmail.com>.
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jo...@gmail.com>.
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Zhihai Xu <zh...@gmail.com>.
Thanks Junping for creating release 2.8.0 (RC3).
+1 (non-binding)
* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.
Thanks
zhihai
On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >> <property>
> >> <name>fs.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >> </property>
> >>
> >> <property>
> >> <name>fs.AbstractFileSystem.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.Adl</value>
> >> </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Zhihai Xu <zh...@gmail.com>.
Thanks Junping for creating release 2.8.0 (RC3).
+1 (non-binding)
* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.
Thanks
zhihai
On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >> <property>
> >> <name>fs.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >> </property>
> >>
> >> <property>
> >> <name>fs.AbstractFileSystem.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.Adl</value>
> >> </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Zhihai Xu <zh...@gmail.com>.
Thanks Junping for creating release 2.8.0 (RC3).
+1 (non-binding)
* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.
Thanks
zhihai
On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >> <property>
> >> <name>fs.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >> </property>
> >>
> >> <property>
> >> <name>fs.AbstractFileSystem.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.Adl</value>
> >> </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Zhihai Xu <zh...@gmail.com>.
Thanks Junping for creating release 2.8.0 (RC3).
+1 (non-binding)
* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.
Thanks
zhihai
On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >> <property>
> >> <name>fs.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >> </property>
> >>
> >> <property>
> >> <name>fs.AbstractFileSystem.adl.impl</name>
> >> <value>org.apache.hadoop.fs.adl.Adl</value>
> >> </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
I'm an =0 I'm afraid
I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of
HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,
of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.
I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.
so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.
Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those
-Steve
> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
>
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>>
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>> <name>fs.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>> <name>fs.AbstractFileSystem.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>>
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>>
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>
>> I am working on a patch.
>>
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding (bin) hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed
Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.
Thanks,
Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
Just a heads up. Looks like some removed the Finish Date off of 2.8.0 in JIRA. It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks again for all who verified and voted!
I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.
Now, we have:
7 binding +1s, from:
Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
Karthik Kambatla, Jian He, Junping Du
18 non-binding +1s, from:
Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
Varun Saxena
1 binding +0, from:
Steve Loughran
and no -1s.
So I am glad to announce that the vote of 2.8.0 RC3 passes.
Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.
I'll push the release bits and send out an announcement for 2.8.0 soon.
Thanks,
Junping
________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks again for all who verified and voted!
I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.
Now, we have:
7 binding +1s, from:
Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
Karthik Kambatla, Jian He, Junping Du
18 non-binding +1s, from:
Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
Varun Saxena
1 binding +0, from:
Steve Loughran
and no -1s.
So I am glad to announce that the vote of 2.8.0 RC3 passes.
Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.
I'll push the release bits and send out an announcement for 2.8.0 soon.
Thanks,
Junping
________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks again for all who verified and voted!
I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.
Now, we have:
7 binding +1s, from:
Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
Karthik Kambatla, Jian He, Junping Du
18 non-binding +1s, from:
Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
Varun Saxena
1 binding +0, from:
Steve Loughran
and no -1s.
So I am glad to announce that the vote of 2.8.0 RC3 passes.
Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.
I'll push the release bits and send out an announcement for 2.8.0 soon.
Thanks,
Junping
________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thanks again for all who verified and voted!
I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.
Now, we have:
7 binding +1s, from:
Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
Karthik Kambatla, Jian He, Junping Du
18 non-binding +1s, from:
Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
Varun Saxena
1 binding +0, from:
Steve Loughran
and no -1s.
So I am glad to announce that the vote of 2.8.0 RC3 passes.
Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.
I'll push the release bits and send out an announcement for 2.8.0 soon.
Thanks,
Junping
________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Karthik Kambatla <ka...@cloudera.com>.
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:
> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > > TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > > With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > > This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > > More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > > New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > > The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > > Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Karthik Kambatla <ka...@cloudera.com>.
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:
> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > > TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > > With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > > This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > > More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > > New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > > The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > > Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Karthik Kambatla <ka...@cloudera.com>.
+1 (binding)
* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI
On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:
> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > > <property>
> > > <name>fs.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > > </property>
> > >
> > > <property>
> > > <name>fs.AbstractFileSystem.adl.impl</name>
> > > <value>org.apache.hadoop.fs.adl.Adl</value>
> > > </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > > TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > > With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > > This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > > More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > > New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > > The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > > Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by "varunsaxena@apache.org" <va...@gmail.com>.
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by "varunsaxena@apache.org" <va...@gmail.com>.
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
I'm an =0 I'm afraid
I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of
HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,
of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.
I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.
so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.
Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those
-Steve
> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
>
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>>
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>> <name>fs.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>> <name>fs.AbstractFileSystem.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>>
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>>
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>
>> I am working on a patch.
>>
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
I'm an =0 I'm afraid
I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of
HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,
of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.
I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.
so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.
Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those
-Steve
> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
>
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>>
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>> <name>fs.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>> <name>fs.AbstractFileSystem.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>>
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>>
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>
>> I am working on a patch.
>>
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by "varunsaxena@apache.org" <va...@gmail.com>.
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
I'm an =0 I'm afraid
I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of
HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,
of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.
I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.
so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.
Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those
-Steve
> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
>
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>>
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>> <name>fs.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>> <name>fs.AbstractFileSystem.adl.impl</name>
>> <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>>
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>>
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>
>> I am working on a patch.
>>
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by "varunsaxena@apache.org" <va...@gmail.com>.
Thanks Junping for creating the release.
+1 (non-binding)
* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.
Regards,
Varun Saxena.
On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:
> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> > <property>
> > <name>fs.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > </property>
> >
> > <property>
> > <name>fs.AbstractFileSystem.adl.impl</name>
> > <value>org.apache.hadoop.fs.adl.Adl</value>
> > </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> > TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> > With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> > This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> > More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> > New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> > The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> > The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> > Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Haibo Chen <ha...@cloudera.com>.
Thanks Junping for working on the new release!
+1 non-binding
1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines
Best,
Haibo Chen
On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Haibo Chen <ha...@cloudera.com>.
Thanks Junping for working on the new release!
+1 non-binding
1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines
Best,
Haibo Chen
On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Haibo Chen <ha...@cloudera.com>.
Thanks Junping for working on the new release!
+1 non-binding
1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines
Best,
Haibo Chen
On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.
Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:
<property>
<name>fs.adl.impl</name>
<value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.adl.impl</name>
<value>org.apache.hadoop.fs.adl.Adl</value>
</property>
I have the initial patch ready but hitting these live unit test failures:
Failed tests:
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>
Tests in error:
TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
Stay tuned...
John Zhuge
Software Engineer, Cloudera
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.
Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:
<property>
<name>fs.adl.impl</name>
<value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.adl.impl</name>
<value>org.apache.hadoop.fs.adl.Adl</value>
</property>
I have the initial patch ready but hitting these live unit test failures:
Failed tests:
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>
Tests in error:
TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
Stay tuned...
John Zhuge
Software Engineer, Cloudera
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jo...@gmail.com>.
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.
Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:
<property>
<name>fs.adl.impl</name>
<value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.adl.impl</name>
<value>org.apache.hadoop.fs.adl.Adl</value>
</property>
I have the initial patch ready but hitting these live unit test failures:
Failed tests:
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>
Tests in error:
TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
Stay tuned...
John Zhuge
Software Engineer, Cloudera
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.
Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:
<property>
<name>fs.adl.impl</name>
<value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.adl.impl</name>
<value>org.apache.hadoop.fs.adl.Adl</value>
</property>
I have the initial patch ready but hitting these live unit test failures:
Failed tests:
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>
Tests in error:
TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
Stay tuned...
John Zhuge
Software Engineer, Cloudera
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jo...@gmail.com>.
+1. Thanks for the great effort, Junping!
- Verified checksums and signatures of the tarballs
- Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
- Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
- Cloud connectors:
- s3a: integration tests, basic fs commands
- adl: live unit tests, basic fs commands. See notes below.
- Deployed a pseudo cluster, passed the following sanity tests in
both insecure and SSL mode:
- HDFS: basic dfs, distcp, ACL commands
- KMS and HttpFS: basic tests
- MapReduce wordcount
- balancer start/stop
Needs the following JIRAs to pass all ADL tests:
- HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
- HDFS-11132. Allow AccessControlException in contract tests when
getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
Dusane
- HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
runtime error. (John Zhuge via lei)
On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
> <property>
> <name>fs.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> </property>
>
> <property>
> <name>fs.AbstractFileSystem.adl.impl</name>
> <value>org.apache.hadoop.fs.adl.Adl</value>
> </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >> With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >> This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >> More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >> New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >> The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >> Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>
--
John
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jz...@cloudera.com>.
Yes, it only affects ADL. There is a workaround of adding these 2
properties to core-site.xml:
<property>
<name>fs.adl.impl</name>
<value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.adl.impl</name>
<value>org.apache.hadoop.fs.adl.Adl</value>
</property>
I have the initial patch ready but hitting these live unit test failures:
Failed tests:
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
expected:<1> but was:<10>
Tests in error:
TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254
» AccessControl
TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190
» AccessControl
Stay tuned...
John Zhuge
Software Engineer, Cloudera
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* John Zhuge <jz...@cloudera.com>
> *Sent:* Monday, March 20, 2017 9:07 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi all,
>> With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>> This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>> More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>> New RC is available at: http://home.apache.org/~junpin
>> g_du/hadoop-2.8.0-RC3
>>
>> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>> The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>> Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for doing this.
*+1 (Binding),*
Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.
Thanks,
Wangda
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for doing this.
*+1 (Binding),*
Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.
Thanks,
Wangda
On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Junping Du <jd...@hortonworks.com>.
Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?
Thanks,
Junping
________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jz...@cloudera.com>.
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
HADOOP-12666 should not be backported, but some changes are needed:
property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Akira Ajisaka <aa...@apache.org>.
Thanks Junping!
+1 (binding)
* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries
successfully
* The document looks good.
I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208
Regards,
Akira
On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked
I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.
Can other people with windows development setups do a run of this?
I've not done any other testing, Windows was enough for a weekend afternoon
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked
I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.
Can other people with windows development setups do a run of this?
I've not done any other testing, Windows was enough for a weekend afternoon
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked
I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.
Can other people with windows development setups do a run of this?
I've not done any other testing, Windows was enough for a weekend afternoon
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked
I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.
Can other people with windows development setups do a run of this?
I've not done any other testing, Windows was enough for a weekend afternoon
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Mingliang Liu <li...@gmail.com>.
Thanks Junping for doing this.
+1 (non-binding)
0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs
L
> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Daniel Templeton <da...@cloudera.com>.
Thanks for the new RC, Junping. I built from source and tried it out on
a 2-node cluster with HA enabled. I ran a pi job and some streaming
jobs. I tested that localization and failover work correctly, and I
played a little with the YARN and HDFS web UIs.
I did encounter an old friend of mine, which is that if you submit a
streaming job with input that is only 1 block, you will nonetheless get
2 mappers that both process the same split. What's new this time is that
the second mapper was consistently failing on certain input sizes. I
(re)verified that the issue also exists is 2.7.3, so it's not a
regression. I'm pretty sure it's been there since at least 2.6.0. I
filed MAPREDUCE-6864 for it.
Given that my issue was not a regression, I'm +1 on the RC.
Daniel
On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Sunil Govind <su...@gmail.com>.
+1 (non-binding). Thanks Junping for the effort.
I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine
Thanks
Sunil
On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Mingliang Liu <li...@gmail.com>.
Thanks Junping for doing this.
+1 (non-binding)
0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs
L
> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by surendra lilhore <su...@huawei.com>.
Hi,
I have used release package and verified basic operations for HDFS.
I raised one blocker issue for this release.
HADOOP-14207 : "dfsadmin -refreshCallQueue" command is failing with DecayRpcScheduler
Thanks,
Surendra
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 14:48
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by John Zhuge <jz...@cloudera.com>.
Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
FileSystem for scheme: adl".
The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
HADOOP-12666 should not be backported, but some changes are needed:
property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
I am working on a patch.
John Zhuge
Software Engineer, Cloudera
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Daniel Templeton <da...@cloudera.com>.
Thanks for the new RC, Junping. I built from source and tried it out on
a 2-node cluster with HA enabled. I ran a pi job and some streaming
jobs. I tested that localization and failover work correctly, and I
played a little with the YARN and HDFS web UIs.
I did encounter an old friend of mine, which is that if you submit a
streaming job with input that is only 1 block, you will nonetheless get
2 mappers that both process the same split. What's new this time is that
the second mapper was consistently failing on certain input sizes. I
(re)verified that the issue also exists is 2.7.3, so it's not a
regression. I'm pretty sure it's been there since at least 2.6.0. I
filed MAPREDUCE-6864 for it.
Given that my issue was not a regression, I'm +1 on the RC.
Daniel
On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Brahma Reddy Battula <br...@huawei.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs
Regards
Brahma Reddy Battula
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Miklos Szegedi <mi...@cloudera.com>.
Hi Junping,
Thank you for working on this.
+1 (Non-Binding)
I verified the following:
1. Deployed on a 3 node cluster with 2 node managers.
2. Configured linux container executor
3. Configured fair scheduler
4. Ran Pi job and verified the results
5. Ran multiple Yarn applications and verified the results
Thank you,
Miklos Szegedi
On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Steve Loughran <st...@hortonworks.com>.
Thanks, testing in progress
here's the windows native binaries to go with it, all signed
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3
The gnu shell commands which came with git worked well here, and the build went through fast, nice to see
Test results to follow
On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Sunil Govind <su...@gmail.com>.
+1 (non-binding). Thanks Junping for the effort.
I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine
Thanks
Sunil
On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
> More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Daniel Templeton <da...@cloudera.com>.
Thanks for the new RC, Junping. I built from source and tried it out on
a 2-node cluster with HA enabled. I ran a pi job and some streaming
jobs. I tested that localization and failover work correctly, and I
played a little with the YARN and HDFS web UIs.
I did encounter an old friend of mine, which is that if you submit a
streaming job with input that is only 1 block, you will nonetheless get
2 mappers that both process the same split. What's new this time is that
the second mapper was consistently failing on certain input sizes. I
(re)verified that the issue also exists is 2.7.3, so it's not a
regression. I'm pretty sure it's been there since at least 2.6.0. I
filed MAPREDUCE-6864 for it.
Given that my issue was not a regression, I'm +1 on the RC.
Daniel
On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for putting this up, Junping.
+1(non-binding)
* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller
context enabled.
Masatake Iwasaki
On 3/17/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Payne <er...@yahoo.com.INVALID>.
+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI
________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Eric Payne <er...@yahoo.com.INVALID>.
+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI
________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Hi all,
With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
Thanks,
Junping
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Mingliang Liu <li...@gmail.com>.
Thanks Junping for doing this.
+1 (non-binding)
0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs
L
> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Daniel Templeton <da...@cloudera.com>.
Thanks for the new RC, Junping. I built from source and tried it out on
a 2-node cluster with HA enabled. I ran a pi job and some streaming
jobs. I tested that localization and failover work correctly, and I
played a little with the YARN and HDFS web UIs.
I did encounter an old friend of mine, which is that if you submit a
streaming job with input that is only 1 block, you will nonetheless get
2 mappers that both process the same split. What's new this time is that
the second mapper was consistently failing on certain input sizes. I
(re)verified that the issue also exists is 2.7.3, so it's not a
regression. I'm pretty sure it's been there since at least 2.6.0. I
filed MAPREDUCE-6864 for it.
Given that my issue was not a regression, I'm +1 on the RC.
Daniel
On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
Posted by Masatake Iwasaki <iw...@oss.nttdata.co.jp>.
Thanks for putting this up, Junping.
+1(non-binding)
* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller
context enabled.
Masatake Iwasaki
On 3/17/17 18:18, Junping Du wrote:
> Hi all,
> With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
> This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
> More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
> New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
> The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
> The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
> Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org