[VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Hi all,
     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.

      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2

      The RC tag in git is: release-2.8.0-RC2

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Steve. That's Awesome! I will kick off a new RC soon.
Shall we reopen HDFS-6200 given issues here? Making it in release note of 2.8.0 could confuse people as it doesn't work in HA deployment.

Thanks,

Junping
________________________________________
From: Steve Loughran
Sent: Thursday, March 16, 2017 7:27 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
>
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
>
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
>
>
> I think we should withdraw the current RC bits. Please stop the verification & vote.
>
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Steve. That's Awesome! I will kick off a new RC soon.
Shall we reopen HDFS-6200 given issues here? Making it in release note of 2.8.0 could confuse people as it doesn't work in HA deployment.

Thanks,

Junping
________________________________________
From: Steve Loughran
Sent: Thursday, March 16, 2017 7:27 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
>
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
>
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
>
>
> I think we should withdraw the current RC bits. Please stop the verification & vote.
>
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Steve. That's Awesome! I will kick off a new RC soon.
Shall we reopen HDFS-6200 given issues here? Making it in release note of 2.8.0 could confuse people as it doesn't work in HA deployment.

Thanks,

Junping
________________________________________
From: Steve Loughran
Sent: Thursday, March 16, 2017 7:27 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
>
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
>
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
>
>
> I think we should withdraw the current RC bits. Please stop the verification & vote.
>
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Steve. That's Awesome! I will kick off a new RC soon.
Shall we reopen HDFS-6200 given issues here? Making it in release note of 2.8.0 could confuse people as it doesn't work in HA deployment.

Thanks,

Junping
________________________________________
From: Steve Loughran
Sent: Thursday, March 16, 2017 7:27 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
>
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
>
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
>
>
> I think we should withdraw the current RC bits. Please stop the verification & vote.
>
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
> 
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
> 
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
> 
> 
> I think we should withdraw the current RC bits. Please stop the verification & vote.
> 
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)-Downloaded source.-Verified signatures.- Compiled the source.-Ran sample jobs like MR sleep on pseudo distributed cluster. (Mac OS)
Thanks Junping and others!Regards,Kuhu
On Wednesday, March 15, 2017, 7:25:46 PM CDT, Junping Du <jd...@hortonworks.com> wrote:bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)-Downloaded source.-Verified signatures.- Compiled the source.-Ran sample jobs like MR sleep on pseudo distributed cluster. (Mac OS)
Thanks Junping and others!Regards,Kuhu
On Wednesday, March 15, 2017, 7:25:46 PM CDT, Junping Du <jd...@hortonworks.com> wrote:bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
> 
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
> 
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
> 
> 
> I think we should withdraw the current RC bits. Please stop the verification & vote.
> 
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)-Downloaded source.-Verified signatures.- Compiled the source.-Ran sample jobs like MR sleep on pseudo distributed cluster. (Mac OS)
Thanks Junping and others!Regards,Kuhu
On Wednesday, March 15, 2017, 7:25:46 PM CDT, Junping Du <jd...@hortonworks.com> wrote:bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
> 
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
> 
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
> 
> 
> I think we should withdraw the current RC bits. Please stop the verification & vote.
> 
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)-Downloaded source.-Verified signatures.- Compiled the source.-Ran sample jobs like MR sleep on pseudo distributed cluster. (Mac OS)
Thanks Junping and others!Regards,Kuhu
On Wednesday, March 15, 2017, 7:25:46 PM CDT, Junping Du <jd...@hortonworks.com> wrote:bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 16 Mar 2017, at 00:25, Junping Du <jd...@hortonworks.com> wrote:
> 
> bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.
> 
> I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.
> 
> 
> I think we should withdraw the current RC bits. Please stop the verification & vote.
> 
> I will kick off another RC immediately when HDFS-11431 get fixed.

is done. hadoop-hdfs without any server-side dependencies is now a hadoop-client dependency.

Release notes:

The hadoop-client POM now includes a leaner hdfs-client, stripping out all the transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself. The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon, xercesImpl, netty and servlet-api.

This should make downstream projects dependent JARs smaller, and avoid version conflict problems with the specific JARs now excluded.

Applications may encounter build problems if they did depend on these JARs, and which didn't explicitly include them. There are two fixes for this

* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing dependencies. For builds against older versions of Hadoop, this will be harmless, as hadoop-hdfs and all its dependencies are already pulled in by the hadoop-client POM.




---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

bq. From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

I could miss that. Thanks for reminding! From my quick check: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-client/2.7.3?, it sounds like 669 artifacts from other projects were depending on it.


I think we should withdraw the current RC bits. Please stop the verification & vote.

I will kick off another RC immediately when HDFS-11431 get fixed.


Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, March 15, 2017 2:04 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping, inline,


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.

From my read of the poms, hadoop-client depends on hadoop-hdfs-client to pull in HDFS-related code. It doesn't have its own dependency on hadoop-hdfs. So I think this affects users of the hadoop-client artifact, which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would affect a lot of users.

I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

Looks like we have a patch being actively revved and reviewed to fix this by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.

Based on the above, I think there is a regression for users of the hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping, inline,

From my understanding, this issue is related to our previous
> improvements with separating client and server jars in HDFS-6200. If we use
> the new "client" jar in NN HA deployment, then we will hit the issue
> reported.
>
From my read of the poms, hadoop-client depends on hadoop-hdfs-client to
pull in HDFS-related code. It doesn't have its own dependency on
hadoop-hdfs. So I think this affects users of the hadoop-client artifact,
which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would
affect a lot of users.

> I can see two options here:
>
> - Without any change in 2.8.0, if user hit the issue when they deploy HA
> cluster by using new client jar, adding back hdfs jar just like how things
> work previously
>
> - Make the change now in 2.8.0, either moving
> ConfiguredFailoverProxyProvider to client jar or adding dependency
> between client jar and server jar. There must be some arguments there on
> which way to fix is better especially ConfiguredFailoverProxyProvider
> still has some sever side dependencies.
>
>
> I would prefer the first option, given:
>
> - The issue fixing time is unpredictable as there are still discussion on
> how to fix this issue. Our 2.8.0 release shouldn't be an endless journey
> which has been deferred several times for more serious issue.
>
Looks like we have a patch being actively revved and reviewed to fix this
by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and
Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

> - We have workaround for this improvement, no regression happens due to
> this issue. People can still use hdfs jar in old way. The worst case
> is improvement for HDFS doesn't work in some cases - that shouldn't block
> the whole release.
>
Based on the above, I think there is a regression for users of the
hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we
can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 00:36, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.
> 
> 
> From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.
> 
> 
> I can see two options here:
> 
> - Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously
> 
> - Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.
> 
> 
> I would prefer the first option, given:
> 
> - The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.
> 
> - We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.
> 
> 
> I think we should let vote keep going unless someone have more concerns which I could miss.

getting it out the door with this in the release notes, and a plan for 2.8.1 would be ideal

> 
> 
> 
> Thanks,
> 
> 
> Junping
> 
> 
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Tuesday, March 14, 2017 2:50 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
> 
> Hi Junping,
> 
> Noticed this possible blocker float by my inbox today. It had an affects but no target version set:
> 
> https://issues.apache.org/jira/browse/HDFS-11431
> 
> Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.
> 
> Best,
> Andrew
> 
> 
> On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping, inline,

From my understanding, this issue is related to our previous
> improvements with separating client and server jars in HDFS-6200. If we use
> the new "client" jar in NN HA deployment, then we will hit the issue
> reported.
>
From my read of the poms, hadoop-client depends on hadoop-hdfs-client to
pull in HDFS-related code. It doesn't have its own dependency on
hadoop-hdfs. So I think this affects users of the hadoop-client artifact,
which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would
affect a lot of users.

> I can see two options here:
>
> - Without any change in 2.8.0, if user hit the issue when they deploy HA
> cluster by using new client jar, adding back hdfs jar just like how things
> work previously
>
> - Make the change now in 2.8.0, either moving
> ConfiguredFailoverProxyProvider to client jar or adding dependency
> between client jar and server jar. There must be some arguments there on
> which way to fix is better especially ConfiguredFailoverProxyProvider
> still has some sever side dependencies.
>
>
> I would prefer the first option, given:
>
> - The issue fixing time is unpredictable as there are still discussion on
> how to fix this issue. Our 2.8.0 release shouldn't be an endless journey
> which has been deferred several times for more serious issue.
>
Looks like we have a patch being actively revved and reviewed to fix this
by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and
Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

> - We have workaround for this improvement, no regression happens due to
> this issue. People can still use hdfs jar in old way. The worst case
> is improvement for HDFS doesn't work in some cases - that shouldn't block
> the whole release.
>
Based on the above, I think there is a regression for users of the
hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we
can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 00:36, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.
> 
> 
> From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.
> 
> 
> I can see two options here:
> 
> - Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously
> 
> - Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.
> 
> 
> I would prefer the first option, given:
> 
> - The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.
> 
> - We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.
> 
> 
> I think we should let vote keep going unless someone have more concerns which I could miss.

getting it out the door with this in the release notes, and a plan for 2.8.1 would be ideal

> 
> 
> 
> Thanks,
> 
> 
> Junping
> 
> 
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Tuesday, March 14, 2017 2:50 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
> 
> Hi Junping,
> 
> Noticed this possible blocker float by my inbox today. It had an affects but no target version set:
> 
> https://issues.apache.org/jira/browse/HDFS-11431
> 
> Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.
> 
> Best,
> Andrew
> 
> 
> On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 00:36, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.
> 
> 
> From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.
> 
> 
> I can see two options here:
> 
> - Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously
> 
> - Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.
> 
> 
> I would prefer the first option, given:
> 
> - The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.
> 
> - We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.
> 
> 
> I think we should let vote keep going unless someone have more concerns which I could miss.

getting it out the door with this in the release notes, and a plan for 2.8.1 would be ideal

> 
> 
> 
> Thanks,
> 
> 
> Junping
> 
> 
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Tuesday, March 14, 2017 2:50 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
> 
> Hi Junping,
> 
> Noticed this possible blocker float by my inbox today. It had an affects but no target version set:
> 
> https://issues.apache.org/jira/browse/HDFS-11431
> 
> Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.
> 
> Best,
> Andrew
> 
> 
> On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping, inline,

From my understanding, this issue is related to our previous
> improvements with separating client and server jars in HDFS-6200. If we use
> the new "client" jar in NN HA deployment, then we will hit the issue
> reported.
>
From my read of the poms, hadoop-client depends on hadoop-hdfs-client to
pull in HDFS-related code. It doesn't have its own dependency on
hadoop-hdfs. So I think this affects users of the hadoop-client artifact,
which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would
affect a lot of users.

> I can see two options here:
>
> - Without any change in 2.8.0, if user hit the issue when they deploy HA
> cluster by using new client jar, adding back hdfs jar just like how things
> work previously
>
> - Make the change now in 2.8.0, either moving
> ConfiguredFailoverProxyProvider to client jar or adding dependency
> between client jar and server jar. There must be some arguments there on
> which way to fix is better especially ConfiguredFailoverProxyProvider
> still has some sever side dependencies.
>
>
> I would prefer the first option, given:
>
> - The issue fixing time is unpredictable as there are still discussion on
> how to fix this issue. Our 2.8.0 release shouldn't be an endless journey
> which has been deferred several times for more serious issue.
>
Looks like we have a patch being actively revved and reviewed to fix this
by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and
Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

> - We have workaround for this improvement, no regression happens due to
> this issue. People can still use hdfs jar in old way. The worst case
> is improvement for HDFS doesn't work in some cases - that shouldn't block
> the whole release.
>
Based on the above, I think there is a regression for users of the
hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we
can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 00:36, Junping Du <jd...@hortonworks.com> wrote:
> 
> Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.
> 
> 
> From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.
> 
> 
> I can see two options here:
> 
> - Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously
> 
> - Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.
> 
> 
> I would prefer the first option, given:
> 
> - The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.
> 
> - We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.
> 
> 
> I think we should let vote keep going unless someone have more concerns which I could miss.

getting it out the door with this in the release notes, and a plan for 2.8.1 would be ideal

> 
> 
> 
> Thanks,
> 
> 
> Junping
> 
> 
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Tuesday, March 14, 2017 2:50 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
> 
> Hi Junping,
> 
> Noticed this possible blocker float by my inbox today. It had an affects but no target version set:
> 
> https://issues.apache.org/jira/browse/HDFS-11431
> 
> Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.
> 
> Best,
> Andrew
> 
> 
> On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping, inline,

From my understanding, this issue is related to our previous
> improvements with separating client and server jars in HDFS-6200. If we use
> the new "client" jar in NN HA deployment, then we will hit the issue
> reported.
>
From my read of the poms, hadoop-client depends on hadoop-hdfs-client to
pull in HDFS-related code. It doesn't have its own dependency on
hadoop-hdfs. So I think this affects users of the hadoop-client artifact,
which has existed for a long time.

Essentially all of our customer deployments run with NN HA, so this would
affect a lot of users.

> I can see two options here:
>
> - Without any change in 2.8.0, if user hit the issue when they deploy HA
> cluster by using new client jar, adding back hdfs jar just like how things
> work previously
>
> - Make the change now in 2.8.0, either moving
> ConfiguredFailoverProxyProvider to client jar or adding dependency
> between client jar and server jar. There must be some arguments there on
> which way to fix is better especially ConfiguredFailoverProxyProvider
> still has some sever side dependencies.
>
>
> I would prefer the first option, given:
>
> - The issue fixing time is unpredictable as there are still discussion on
> how to fix this issue. Our 2.8.0 release shouldn't be an endless journey
> which has been deferred several times for more serious issue.
>
Looks like we have a patch being actively revved and reviewed to fix this
by making hadoop-hdfs-client depend on hadoop-hdfs. Thanks to Steven and
Steve for working on this.

Steve proposed doing a proper split in a later JIRA.

> - We have workaround for this improvement, no regression happens due to
> this issue. People can still use hdfs jar in old way. The worst case
> is improvement for HDFS doesn't work in some cases - that shouldn't block
> the whole release.
>
Based on the above, I think there is a regression for users of the
hadoop-client artifact.

If it actually only affects users of hadoop-hdfs-client, then I agree we
can document it as a Known Issue and fix it later.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.


I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.


I think we should let vote keep going unless someone have more concerns which I could miss.



Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Tuesday, March 14, 2017 2:50 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.

      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2

      The RC tag in git is: release-2.8.0-RC2

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.


I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.


I think we should let vote keep going unless someone have more concerns which I could miss.



Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Tuesday, March 14, 2017 2:50 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.

      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2

      The RC tag in git is: release-2.8.0-RC2

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.


I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.


I think we should let vote keep going unless someone have more concerns which I could miss.



Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Tuesday, March 14, 2017 2:50 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.

      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2

      The RC tag in git is: release-2.8.0-RC2

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Thanks Andrew for reporting the issue. This JIRA is out of my radar as it? didn't specify any target version before.


From my understanding, this issue is related to our previous improvements with separating client and server jars in HDFS-6200. If we use the new "client" jar in NN HA deployment, then we will hit the issue reported.


I can see two options here:

- Without any change in 2.8.0, if user hit the issue when they deploy HA cluster by using new client jar, adding back hdfs jar just like how things work previously

- Make the change now in 2.8.0, either moving ConfiguredFailoverProxyProvider to client jar or adding dependency between client jar and server jar. There must be some arguments there on which way to fix is better especially ConfiguredFailoverProxyProvider still has some sever side dependencies.


I would prefer the first option, given:

- The issue fixing time is unpredictable as there are still discussion on how to fix this issue. Our 2.8.0 release shouldn't be an endless journey which has been deferred several times for more serious issue.

- We have workaround for this improvement, no regression happens due to this issue. People can still use hdfs jar in old way. The worst case is improvement for HDFS doesn't work in some cases - that shouldn't block the whole release.


I think we should let vote keep going unless someone have more concerns which I could miss.



Thanks,


Junping


________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Tuesday, March 14, 2017 2:50 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.

      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2

      The RC tag in git is: release-2.8.0-RC2

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1056

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects
but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With several important fixes get merged last week, I've created a new
> release candidate (RC2) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,919 fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       Please note that RC0 and RC1 are not voted public because
> significant issues are found just after RC tag getting published.
>
>       The RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC2
>
>       The RC tag in git is: release-2.8.0-RC2
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1056
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects
but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With several important fixes get merged last week, I've created a new
> release candidate (RC2) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,919 fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       Please note that RC0 and RC1 are not voted public because
> significant issues are found just after RC tag getting published.
>
>       The RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC2
>
>       The RC tag in git is: release-2.8.0-RC2
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1056
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Vinod Kumar Vavilapalli <vi...@apache.org>]

Thanks for taking the mantle from me on 2.8.0 and some persistent work getting 2.8.0 out the door, Junping!

Apologies for bringing this up late, but I’d like to add one comment.

We should repeat what we did for 2.7.0 and In line with our experience there, we should annotate this release as not ready for production use. See the releases page - http://hadoop.apache.org/releases.html#25+August%2C+2016%3A+Release+2.7.3+available for our messaging on 2.7.0.

The expectation is that more downstream projects pick up the bits, iron out any incompatibilities we might have missed, and production users then pick up a solid 2.8.1.

Thanks
+Vinod

> On Mar 14, 2017, at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Fwd: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

There's an RC of Hadoop 2.8.0 out for people to build and test against. This is the last chance for people to find problems and complain before it gets out the door

Spark happily builds against it. my tests all work. For anyone wanting to do the build, the maven arguments are: -Phive,yarn,hadoop-2.7 -Dhadoop.version=2.8.0 -Psnapshots-and-staging

One thing I would be curious about is how well things work on Windows, especially how informative it is if things like winutils.exe can't be found. Previous versions: stack trace on startup, even if not relevant, and meaningless text on use. With HADOOP-10775, stack traces should be postponed until point of use, and error text including a ref to : https://wiki.apache.org/hadoop/WindowsProblems <https://wiki.apache.org/hadoop/WindowsProblems>

I would really like to know how well that works for people.

Here's the win 64 native binaries to go with the ASF artifacts; built off the same commit
https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3 <https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3>


> Begin forwarded message:
> 
> From: Junping Du <jd...@hortonworks.com>
> Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> Date: 17 March 2017 at 09:18:15 GMT
> To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>, "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
> 
> Hi all,
>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
> 
>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
> 
> Thanks,
> 
> Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jason Lowe <jl...@yahoo-inc.com.INVALID>]

+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
 

    On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
 

 Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

   

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Brahma Reddy Battula <br...@huawei.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs

Regards
Brahma Reddy Battula

-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Brahma Reddy Battula <br...@huawei.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs

Regards
Brahma Reddy Battula

-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Marton Elek <me...@hortonworks.com>]

+1 (non-binding)

Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers) 
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS

Looks good.

Marton


> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
> 
> +1 (non-binding). Thanks Junping for the effort.
> 
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
> 
> Thanks
> Sunil
> 
> 
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> 
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Marton Elek <me...@hortonworks.com>]

+1 (non-binding)

Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers) 
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS

Looks good.

Marton


> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
> 
> +1 (non-binding). Thanks Junping for the effort.
> 
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
> 
> Thanks
> Sunil
> 
> 
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> 
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Marton Elek <me...@hortonworks.com>]

+1 (non-binding)

Tested from the released binary package
* 5 node cluster running from dockerized containers (every namenode/datanode/nodemanager, etc. are running in separated containers) 
* Bitcoin bockchain data (~100Gb) parsed and imported to HBase (1.2.4)
* Spark (2.1.0 with included hadoop) job (executing on YARN) to query the data from HBase and write the results to HDFS

Looks good.

Marton


> On Mar 19, 2017, at 6:01 PM, Sunil Govind <su...@gmail.com> wrote:
> 
> +1 (non-binding). Thanks Junping for the effort.
> 
> I have used release package and verified below cases
> - Ran MR sleep job and wordcount successfully in where nodes are configured
> with labels.
> - Verified application priority feature and I could see high priority apps
> are getting resource over lower priority apps when configured
> - Verified RM web UI pages and looks fine (priority could be seen)
> - Intra-queue preemption related to app priority also seems fine
> 
> Thanks
> Sunil
> 
> 
> On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:
> 
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at:
>> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Sunil Govind <su...@gmail.com>]

+1 (non-binding). Thanks Junping for the effort.

I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine

Thanks
Sunil


On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Mingliang Liu <li...@gmail.com>]

Thanks Junping for doing this.

+1 (non-binding)

0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs

L

> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
> 
>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
> 
> Thanks,
> 
> Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jason Lowe <jl...@yahoo-inc.com.INVALID>]

+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
 

    On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
 

 Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

   

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Sunil Govind <su...@gmail.com>]

+1 (non-binding). Thanks Junping for the effort.

I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine

Thanks
Sunil


On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[surendra lilhore <su...@huawei.com>]

Hi,
	I have used release package and verified basic operations for HDFS.
	I raised one blocker issue for this release.

	HADOOP-14207 : "dfsadmin -refreshCallQueue" command is failing with DecayRpcScheduler
	
Thanks,
Surendra
	
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 14:48
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[surendra lilhore <su...@huawei.com>]

Hi,
	I have used release package and verified basic operations for HDFS.
	I raised one blocker issue for this release.

	HADOOP-14207 : "dfsadmin -refreshCallQueue" command is failing with DecayRpcScheduler
	
Thanks,
Surendra
	
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 14:48
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Brahma Reddy Battula <br...@huawei.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs

Regards
Brahma Reddy Battula

-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jason Lowe <jl...@yahoo-inc.com.INVALID>]

+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
 

    On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
 

 Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

   

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Miklos Szegedi <mi...@cloudera.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

I verified the following:
1. Deployed on a 3 node cluster with 2 node managers.
2. Configured linux container executor
3. Configured fair scheduler
4. Ran Pi job and verified the results
5. Ran multiple Yarn applications and verified the results

Thank you,
Miklos Szegedi

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Payne <er...@yahoo.com.INVALID>]

+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI




________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org> 
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)



Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.


Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.


Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process.  So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot.  Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.


Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process.  So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot.  Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.


Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process.  So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot.  Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Yufei Gu <fl...@gmail.com>]

Thank Junping for working on this.

I verified the following:
1. Verified the md5 of binary tar ball.
2. Deployed on a 4 node cluster with 3 node managers.
3. Configured fair scheduler
4. Ran Pi job and verified the results.
5. Ran SLS for fair scheduler and capacity scheduler.

All good except SLS, it doesn't work. I guess this is an known issue.

Best,

Yufei

On Tue, Mar 21, 2017 at 10:12 AM, Andrew Wang <an...@cloudera.com>
wrote:

> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.
>
> It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
> extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
> content:
>
> -> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
> {print SUM}'
> 1651891481
>
>
> On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
> > Thanks Junping for taking on this huge effort!
> >
> > I found one tiny nit: the md5 files are not in conventional format
> > That is to say,
> > $ cat hadoop-2.8.0.tar.gz.md5
> >
> > $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> > c728a090b68d009070085367695ed507  /build/source/target/
> > artifacts/hadoop-2.8.0.tar.gz
> >
> > But a typical md5 file would have been:
> > c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz
> >
> > I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> > binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> > But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> > comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> > amount of changes made into Hadoop 2.8 makes such difference in size.
> >
> > Regards,
> > Wei-Chiu Chuang
> >
> > > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org>
> wrote:
> > >
> > > Thanks Junping!
> > >
> > > +1 (binding)
> > >
> > > * Verified signatures and checksums
> > > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > > * Deployed a single node cluster and ran some Hive on Tez queries
> > successfully
> > > * The document looks good.
> > >
> > > I found a trivial issue in the doc. It does not block the release.
> > > https://issues.apache.org/jira/browse/HADOOP-14208
> > >
> > > Regards,
> > > Akira
> > >
> > >
> > > On 2017/03/17 18:18, Junping Du wrote:
> > >> Hi all,
> > >>     With fix of HDFS-11431 get in, I've created a new release
> candidate
> > (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>     This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > https://repository.apache.org/content/repositories/orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > days, ending on 03/22/2017 PDT time.
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

> On Mar 21, 2017, at 10:12 AM, Andrew Wang <an...@cloudera.com> wrote:
> 
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.


Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process.  So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot.  Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Andrew Wang <an...@cloudera.com>]

I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.

It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:

-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481


On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507  /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Andrew Wang <an...@cloudera.com>]

I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.

It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:

-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481


On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507  /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Andrew Wang <an...@cloudera.com>]

I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.

It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:

-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481


On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507  /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Andrew Wang <an...@cloudera.com>]

I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
more changes than 2.8.0.

It looks like the 2.8.0 bin tarball has an extra 1.5GB of docs when
extracted compared to 3.0.0-alpha2. I think it's from the extra src-html
content:

-> % find share/doc -name src-html | xargs du -sb | awk -e '{SUM+=$1} END
{print SUM}'
1651891481


On Tue, Mar 21, 2017 at 9:53 AM, Wei-Chiu Chuang <we...@cloudera.com>
wrote:

> Thanks Junping for taking on this huge effort!
>
> I found one tiny nit: the md5 files are not in conventional format
> That is to say,
> $ cat hadoop-2.8.0.tar.gz.md5
>
> $ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
> c728a090b68d009070085367695ed507  /build/source/target/
> artifacts/hadoop-2.8.0.tar.gz
>
> But a typical md5 file would have been:
> c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz
>
> I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB
> binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
> But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB
> comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the
> amount of changes made into Hadoop 2.8 makes such difference in size.
>
> Regards,
> Wei-Chiu Chuang
>
> > On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> >
> > Thanks Junping!
> >
> > +1 (binding)
> >
> > * Verified signatures and checksums
> > * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> > * Deployed a single node cluster and ran some Hive on Tez queries
> successfully
> > * The document looks good.
> >
> > I found a trivial issue in the doc. It does not block the release.
> > https://issues.apache.org/jira/browse/HADOOP-14208
> >
> > Regards,
> > Akira
> >
> >
> > On 2017/03/17 18:18, Junping Du wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wei-Chiu Chuang <we...@cloudera.com>]

Thanks Junping for taking on this huge effort!

I found one tiny nit: the md5 files are not in conventional format
That is to say, 
$ cat hadoop-2.8.0.tar.gz.md5 

$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507  /build/source/target/artifacts/hadoop-2.8.0.tar.gz

But a typical md5 file would have been:
c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz

I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.

Regards,
Wei-Chiu Chuang

> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> 
> Thanks Junping!
> 
> +1 (binding)
> 
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
> 
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
> 
> Regards,
> Akira
> 
> 
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wei-Chiu Chuang <we...@cloudera.com>]

Thanks Junping for taking on this huge effort!

I found one tiny nit: the md5 files are not in conventional format
That is to say, 
$ cat hadoop-2.8.0.tar.gz.md5 

$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507  /build/source/target/artifacts/hadoop-2.8.0.tar.gz

But a typical md5 file would have been:
c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz

I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.

Regards,
Wei-Chiu Chuang

> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> 
> Thanks Junping!
> 
> +1 (binding)
> 
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
> 
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
> 
> Regards,
> Akira
> 
> 
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wei-Chiu Chuang <we...@cloudera.com>]

Thanks Junping for taking on this huge effort!

I found one tiny nit: the md5 files are not in conventional format
That is to say, 
$ cat hadoop-2.8.0.tar.gz.md5 

$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507  /build/source/target/artifacts/hadoop-2.8.0.tar.gz

But a typical md5 file would have been:
c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz

I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.

Regards,
Wei-Chiu Chuang

> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> 
> Thanks Junping!
> 
> +1 (binding)
> 
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
> 
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
> 
> Regards,
> Akira
> 
> 
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wei-Chiu Chuang <we...@cloudera.com>]

Thanks Junping for taking on this huge effort!

I found one tiny nit: the md5 files are not in conventional format
That is to say, 
$ cat hadoop-2.8.0.tar.gz.md5 

$ /usr/bin/md5sum /build/source/target/artifacts/hadoop-2.8.0.tar.gz
c728a090b68d009070085367695ed507  /build/source/target/artifacts/hadoop-2.8.0.tar.gz

But a typical md5 file would have been:
c728a090b68d009070085367695ed507  hadoop-2.8.0.tar.gz

I was pretty stunned finding hadoop-2.8.0.tar.gz is a whopping 410MB binary, comparing to hadoop-2.7.3.tar.gz which is just 205 MB.
But later on I realized the source code hadoop-2.8.0-src.tar.gz is 33MB comparing to hadoop-2.7.3-src.tar.gz which is 18MB. So probably it’s the amount of changes made into Hadoop 2.8 makes such difference in size.

Regards,
Wei-Chiu Chuang

> On Mar 21, 2017, at 9:39 AM, Akira Ajisaka <aa...@apache.org> wrote:
> 
> Thanks Junping!
> 
> +1 (binding)
> 
> * Verified signatures and checksums
> * Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
> * Deployed a single node cluster and ran some Hive on Tez queries successfully
> * The document looks good.
> 
> I found a trivial issue in the doc. It does not block the release.
> https://issues.apache.org/jira/browse/HADOOP-14208
> 
> Regards,
> Akira
> 
> 
> On 2017/03/17 18:18, Junping Du wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Akira Ajisaka <aa...@apache.org>]

Thanks Junping!

+1 (binding)

* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries 
successfully
* The document looks good.

I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208

Regards,
Akira


On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>       More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Gergo Pasztor <pa...@cloudera.com>]

+1 (non-binding)

- verified signatures
- run hadoop mapreduce examples
- use hdfs through shell

Best regards,
Gergő Pásztor


On Tue, Mar 21, 2017 at 5:17 PM, Masatake Iwasaki <
iwasakims@oss.nttdata.co.jp> wrote:

> Thanks for putting this up, Junping.
>
> +1(non-binding)
>
> * verified signature and mds.
> * built from source on centos 7 and openjdk 8 with native profile.
> * built docs and skimmed the contents.
> * deployed 3 nodes ha cluster and run some sample jobs with caller context
> enabled.
>
> Masatake Iwasaki
>
>
> On 3/17/17 18:18, Junping Du wrote:
>
>> Hi all,
>>       With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>>       This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>>        More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>>        New RC is available at: http://home.apache.org/~junpin
>> g_du/hadoop-2.8.0-RC3
>>
>>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>>        The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>>        Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Masatake Iwasaki <iw...@oss.nttdata.co.jp>]

Thanks for putting this up, Junping.

+1(non-binding)

* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller 
context enabled.

Masatake Iwasaki

On 3/17/17 18:18, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Masatake Iwasaki <iw...@oss.nttdata.co.jp>]

Thanks for putting this up, Junping.

+1(non-binding)

* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller 
context enabled.

Masatake Iwasaki

On 3/17/17 18:18, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Akira Ajisaka <aa...@apache.org>]

Thanks Junping!

+1 (binding)

* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries 
successfully
* The document looks good.

I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208

Regards,
Akira


On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>       More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Mingliang Liu <li...@gmail.com>]

Thanks Junping for doing this.

+1 (non-binding)

0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs

L

> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
> 
>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
> 
> Thanks,
> 
> Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Akira Ajisaka <aa...@apache.org>]

Thanks Junping!

+1 (binding)

* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries 
successfully
* The document looks good.

I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208

Regards,
Akira


On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>       More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wangda Tan <wh...@gmail.com>]

Thanks Junping for doing this.

*+1 (Binding),*

Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.

Thanks,
Wangda


On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:

> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wangda Tan <wh...@gmail.com>]

Thanks Junping for doing this.

*+1 (Binding),*

Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.

Thanks,
Wangda


On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:

> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Naganarasimha Garla <na...@apache.org>]

Thanks Junping for putting this up,.

+1 (non-binding)

* downloaded verified signature and md5.
* Built from source & deployed  pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.

Regards,
+ Naga



On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:

> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>>  - Verified checksums and signatures of the tarballs
> > >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>>  - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>>  - Cloud connectors:
> > >>>      - s3a: integration tests, basic fs commands
> > >>>      - adl: live unit tests, basic fs commands. See notes below.
> > >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> > >>>  both insecure and SSL mode:
> > >>>      - HDFS: basic dfs, distcp, ACL commands
> > >>>      - KMS and HttpFS: basic tests
> > >>>      - MapReduce wordcount
> > >>>      - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> > >>>  getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>>  Dusane
> > >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>>  runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>>     This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>>     More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>>     New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>>     The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>>     Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Naganarasimha Garla <na...@apache.org>]

Thanks Junping for putting this up,.

+1 (non-binding)

* downloaded verified signature and md5.
* Built from source & deployed  pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.

Regards,
+ Naga



On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:

> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>>  - Verified checksums and signatures of the tarballs
> > >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>>  - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>>  - Cloud connectors:
> > >>>      - s3a: integration tests, basic fs commands
> > >>>      - adl: live unit tests, basic fs commands. See notes below.
> > >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> > >>>  both insecure and SSL mode:
> > >>>      - HDFS: basic dfs, distcp, ACL commands
> > >>>      - KMS and HttpFS: basic tests
> > >>>      - MapReduce wordcount
> > >>>      - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> > >>>  getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>>  Dusane
> > >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>>  runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>>     This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>>     More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>>     New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>>     The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>>     Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Naganarasimha Garla <na...@apache.org>]

Thanks Junping for putting this up,.

+1 (non-binding)

* downloaded verified signature and md5.
* Built from source & deployed  pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.

Regards,
+ Naga



On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:

> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>>  - Verified checksums and signatures of the tarballs
> > >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>>  - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>>  - Cloud connectors:
> > >>>      - s3a: integration tests, basic fs commands
> > >>>      - adl: live unit tests, basic fs commands. See notes below.
> > >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> > >>>  both insecure and SSL mode:
> > >>>      - HDFS: basic dfs, distcp, ACL commands
> > >>>      - KMS and HttpFS: basic tests
> > >>>      - MapReduce wordcount
> > >>>      - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> > >>>  getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>>  Dusane
> > >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>>  runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>>     This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>>     More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>>     New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>>     The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>>     Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Naganarasimha Garla <na...@apache.org>]

Thanks Junping for putting this up,.

+1 (non-binding)

* downloaded verified signature and md5.
* Built from source & deployed  pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.

Regards,
+ Naga



On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <ra...@apache.org>
wrote:

> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> > hdfs-dev@hadoop.apache.org
> > >> ;
> > >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <eb...@yahoo-inc.com.INVALID> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <john.zhuge@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>>  - Verified checksums and signatures of the tarballs
> > >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>>  - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>>  - Cloud connectors:
> > >>>      - s3a: integration tests, basic fs commands
> > >>>      - adl: live unit tests, basic fs commands. See notes below.
> > >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> > >>>  both insecure and SSL mode:
> > >>>      - HDFS: basic dfs, distcp, ACL commands
> > >>>      - KMS and HttpFS: basic tests
> > >>>      - MapReduce wordcount
> > >>>      - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> > >>>  getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>>  Dusane
> > >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>>  runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <jz...@cloudera.com>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>>     This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>>     More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>>     New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>>     The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>>     Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >>>
> > >>
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Rakesh Radhakrishnan <ra...@apache.org>]

Thanks Junping for getting this out.

+1 (non-binding)

* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds


Rakesh

On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:

> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>>  - Verified checksums and signatures of the tarballs
> >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>>  - Cloud connectors:
> >>>      - s3a: integration tests, basic fs commands
> >>>      - adl: live unit tests, basic fs commands. See notes below.
> >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> >>>  both insecure and SSL mode:
> >>>      - HDFS: basic dfs, distcp, ACL commands
> >>>      - KMS and HttpFS: basic tests
> >>>      - MapReduce wordcount
> >>>      - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> >>>  getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>>  Dusane
> >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>>  runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>>   <name>fs.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>>     This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>>     More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>>     New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>>     The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>>     Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Rakesh Radhakrishnan <ra...@apache.org>]

Thanks Junping for getting this out.

+1 (non-binding)

* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds


Rakesh

On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:

> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>>  - Verified checksums and signatures of the tarballs
> >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>>  - Cloud connectors:
> >>>      - s3a: integration tests, basic fs commands
> >>>      - adl: live unit tests, basic fs commands. See notes below.
> >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> >>>  both insecure and SSL mode:
> >>>      - HDFS: basic dfs, distcp, ACL commands
> >>>      - KMS and HttpFS: basic tests
> >>>      - MapReduce wordcount
> >>>      - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> >>>  getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>>  Dusane
> >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>>  runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>>   <name>fs.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>>     This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>>     More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>>     New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>>     The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>>     Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Rakesh Radhakrishnan <ra...@apache.org>]

Thanks Junping for getting this out.

+1 (non-binding)

* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds


Rakesh

On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:

> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>>  - Verified checksums and signatures of the tarballs
> >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>>  - Cloud connectors:
> >>>      - s3a: integration tests, basic fs commands
> >>>      - adl: live unit tests, basic fs commands. See notes below.
> >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> >>>  both insecure and SSL mode:
> >>>      - HDFS: basic dfs, distcp, ACL commands
> >>>      - KMS and HttpFS: basic tests
> >>>      - MapReduce wordcount
> >>>      - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> >>>  getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>>  Dusane
> >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>>  runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>>   <name>fs.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>>     This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>>     More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>>     New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>>     The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>>     Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Rakesh Radhakrishnan <ra...@apache.org>]

Thanks Junping for getting this out.

+1 (non-binding)

* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds


Rakesh

On Wed, Mar 22, 2017 at 8:30 PM, Jian He <jh...@hortonworks.com> wrote:

> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <ks...@yahoo-inc.com>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org
> >> ;
> >>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <eb...@yahoo-inc.com.INVALID> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>>  - Verified checksums and signatures of the tarballs
> >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>>  - Cloud connectors:
> >>>      - s3a: integration tests, basic fs commands
> >>>      - adl: live unit tests, basic fs commands. See notes below.
> >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> >>>  both insecure and SSL mode:
> >>>      - HDFS: basic dfs, distcp, ACL commands
> >>>      - KMS and HttpFS: basic tests
> >>>      - MapReduce wordcount
> >>>      - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> >>>  getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>>  Dusane
> >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>>  runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>>   <name>fs.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <jz...@cloudera.com>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>>     This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>>     More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>>     New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>>     The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>>     Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jian He <jh...@hortonworks.com>]

+1 (binding)

- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.

Jian

> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> 
> +1 (non-binding)
> 
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
> 
> 
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> 
>> Thanks for all the effort Junping!
>> 
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>> 
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>> 
>> Thanks
>> Ravi
>> 
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>> 
>>> Thanks all for response with verification work and vote!
>>> 
>>> 
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>> 
>>> 
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> Junping
>>> 
>>> 
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>> 
>>> 
>>> +1 (non-binding)
>>> 
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>> 
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>> 
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>> 
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>> 
>>> Thanks,
>>> 
>>> Eric
>>> 
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>> 
>>> 
>>> 
>>> +1. Thanks for the great effort, Junping!
>>> 
>>> 
>>>  - Verified checksums and signatures of the tarballs
>>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>>  - Cloud connectors:
>>>      - s3a: integration tests, basic fs commands
>>>      - adl: live unit tests, basic fs commands. See notes below.
>>>  - Deployed a pseudo cluster, passed the following sanity tests in
>>>  both insecure and SSL mode:
>>>      - HDFS: basic dfs, distcp, ACL commands
>>>      - KMS and HttpFS: basic tests
>>>      - MapReduce wordcount
>>>      - balancer start/stop
>>> 
>>> 
>>> Needs the following JIRAs to pass all ADL tests:
>>> 
>>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>>  - HDFS-11132. Allow AccessControlException in contract tests when
>>>  getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>>  Dusane
>>>  - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>>  runtime error. (John Zhuge via lei)
>>> 
>>> 
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>> 
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>> 
>>>> <property>
>>>>   <name>fs.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>> 
>>>> <property>
>>>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>> 
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>> 
>>>> Failed tests:
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>> 
>>>> Tests in error:
>>>> 
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>> 
>>>> 
>>>> Stay tuned...
>>>> 
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>> 
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>> 
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> 
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>> 
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>> 
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>> 
>>>>> I am working on a patch.
>>>>> 
>>>>> 
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>> 
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>> 
>>>>>> Hi all,
>>>>>>     With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>> 
>>>>>>     This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>> 
>>>>>>     More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>> 
>>>>>>     New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>> 
>>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>> 
>>>>>>     The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>> 
>>>>>>     Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Junping
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> John
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> 
>> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jian He <jh...@hortonworks.com>]

+1 (binding)

- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.

Jian

> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> 
> +1 (non-binding)
> 
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
> 
> 
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> 
>> Thanks for all the effort Junping!
>> 
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>> 
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>> 
>> Thanks
>> Ravi
>> 
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>> 
>>> Thanks all for response with verification work and vote!
>>> 
>>> 
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>> 
>>> 
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> Junping
>>> 
>>> 
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>> 
>>> 
>>> +1 (non-binding)
>>> 
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>> 
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>> 
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>> 
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>> 
>>> Thanks,
>>> 
>>> Eric
>>> 
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>> 
>>> 
>>> 
>>> +1. Thanks for the great effort, Junping!
>>> 
>>> 
>>>  - Verified checksums and signatures of the tarballs
>>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>>  - Cloud connectors:
>>>      - s3a: integration tests, basic fs commands
>>>      - adl: live unit tests, basic fs commands. See notes below.
>>>  - Deployed a pseudo cluster, passed the following sanity tests in
>>>  both insecure and SSL mode:
>>>      - HDFS: basic dfs, distcp, ACL commands
>>>      - KMS and HttpFS: basic tests
>>>      - MapReduce wordcount
>>>      - balancer start/stop
>>> 
>>> 
>>> Needs the following JIRAs to pass all ADL tests:
>>> 
>>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>>  - HDFS-11132. Allow AccessControlException in contract tests when
>>>  getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>>  Dusane
>>>  - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>>  runtime error. (John Zhuge via lei)
>>> 
>>> 
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>> 
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>> 
>>>> <property>
>>>>   <name>fs.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>> 
>>>> <property>
>>>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>> 
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>> 
>>>> Failed tests:
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>> 
>>>> Tests in error:
>>>> 
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>> 
>>>> 
>>>> Stay tuned...
>>>> 
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>> 
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>> 
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> 
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>> 
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>> 
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>> 
>>>>> I am working on a patch.
>>>>> 
>>>>> 
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>> 
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>> 
>>>>>> Hi all,
>>>>>>     With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>> 
>>>>>>     This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>> 
>>>>>>     More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>> 
>>>>>>     New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>> 
>>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>> 
>>>>>>     The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>> 
>>>>>>     Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Junping
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> John
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> 
>> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jian He <jh...@hortonworks.com>]

+1 (binding)

- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.

Jian

> On Mar 22, 2017, at 9:03 PM, larry mccay <la...@gmail.com> wrote:
> 
> +1 (non-binding)
> 
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
> 
> 
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:
> 
>> Thanks for all the effort Junping!
>> 
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>> 
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>> 
>> Thanks
>> Ravi
>> 
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>> 
>>> Thanks all for response with verification work and vote!
>>> 
>>> 
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>> 
>>> 
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> Junping
>>> 
>>> 
>>> ________________________________
>>> From: Kuhu Shukla <ks...@yahoo-inc.com>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
>> ;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>> 
>>> 
>>> +1 (non-binding)
>>> 
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>> 
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>> 
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <eb...@yahoo-inc.com.INVALID> wrote:
>>> +1 (non-binding)
>>> 
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>> 
>>> Thanks,
>>> 
>>> Eric
>>> 
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
>>> wrote:
>>> 
>>> 
>>> 
>>> +1. Thanks for the great effort, Junping!
>>> 
>>> 
>>>  - Verified checksums and signatures of the tarballs
>>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>>  - Cloud connectors:
>>>      - s3a: integration tests, basic fs commands
>>>      - adl: live unit tests, basic fs commands. See notes below.
>>>  - Deployed a pseudo cluster, passed the following sanity tests in
>>>  both insecure and SSL mode:
>>>      - HDFS: basic dfs, distcp, ACL commands
>>>      - KMS and HttpFS: basic tests
>>>      - MapReduce wordcount
>>>      - balancer start/stop
>>> 
>>> 
>>> Needs the following JIRAs to pass all ADL tests:
>>> 
>>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>>  - HDFS-11132. Allow AccessControlException in contract tests when
>>>  getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>>  Dusane
>>>  - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>>  runtime error. (John Zhuge via lei)
>>> 
>>> 
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
>> wrote:
>>> 
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>> 
>>>> <property>
>>>>   <name>fs.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>> 
>>>> <property>
>>>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>> 
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>> 
>>>> Failed tests:
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>> 
>>>> Tests in error:
>>>> 
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>> 
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>> 
>>>> 
>>>> Stay tuned...
>>>> 
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>> 
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>> 
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> 
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <jz...@cloudera.com>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>> 
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>> 
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>> 
>>>>> I am working on a patch.
>>>>> 
>>>>> 
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>> 
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
>>> wrote:
>>>>> 
>>>>>> Hi all,
>>>>>>     With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>> 
>>>>>>     This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>> 
>>>>>>     More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>> 
>>>>>>     New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>> 
>>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>> 
>>>>>>     The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>> 
>>>>>>     Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>> 
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Junping
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> John
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> 
>> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[larry mccay <la...@gmail.com>]

+1 (non-binding)

- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox


On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:

> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> >   - Verified checksums and signatures of the tarballs
> >   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >   - Cloud connectors:
> >       - s3a: integration tests, basic fs commands
> >       - adl: live unit tests, basic fs commands. See notes below.
> >   - Deployed a pseudo cluster, passed the following sanity tests in
> >   both insecure and SSL mode:
> >       - HDFS: basic dfs, distcp, ACL commands
> >       - KMS and HttpFS: basic tests
> >       - MapReduce wordcount
> >       - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> >   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> >   - HDFS-11132. Allow AccessControlException in contract tests when
> >   getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> >   Dusane
> >   - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> >   runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >  <property>
> > >    <name>fs.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >  </property>
> > >
> > >  <property>
> > >    <name>fs.AbstractFileSystem.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.Adl</value>
> > >  </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>      With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >>      This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >>      More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >>      New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >>      The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >>      Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[larry mccay <la...@gmail.com>]

+1 (non-binding)

- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox


On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:

> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> >   - Verified checksums and signatures of the tarballs
> >   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >   - Cloud connectors:
> >       - s3a: integration tests, basic fs commands
> >       - adl: live unit tests, basic fs commands. See notes below.
> >   - Deployed a pseudo cluster, passed the following sanity tests in
> >   both insecure and SSL mode:
> >       - HDFS: basic dfs, distcp, ACL commands
> >       - KMS and HttpFS: basic tests
> >       - MapReduce wordcount
> >       - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> >   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> >   - HDFS-11132. Allow AccessControlException in contract tests when
> >   getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> >   Dusane
> >   - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> >   runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >  <property>
> > >    <name>fs.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >  </property>
> > >
> > >  <property>
> > >    <name>fs.AbstractFileSystem.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.Adl</value>
> > >  </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>      With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >>      This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >>      More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >>      New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >>      The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >>      Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[larry mccay <la...@gmail.com>]

+1 (non-binding)

- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox


On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:

> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> >   - Verified checksums and signatures of the tarballs
> >   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >   - Cloud connectors:
> >       - s3a: integration tests, basic fs commands
> >       - adl: live unit tests, basic fs commands. See notes below.
> >   - Deployed a pseudo cluster, passed the following sanity tests in
> >   both insecure and SSL mode:
> >       - HDFS: basic dfs, distcp, ACL commands
> >       - KMS and HttpFS: basic tests
> >       - MapReduce wordcount
> >       - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> >   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> >   - HDFS-11132. Allow AccessControlException in contract tests when
> >   getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> >   Dusane
> >   - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> >   runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >  <property>
> > >    <name>fs.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >  </property>
> > >
> > >  <property>
> > >    <name>fs.AbstractFileSystem.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.Adl</value>
> > >  </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>      With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >>      This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >>      More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >>      New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >>      The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >>      Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[larry mccay <la...@gmail.com>]

+1 (non-binding)

- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox


On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <ra...@gmail.com> wrote:

> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <ks...@yahoo-inc.com>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org
> ;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <eb...@yahoo-inc.com.INVALID> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> >   - Verified checksums and signatures of the tarballs
> >   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >   - Cloud connectors:
> >       - s3a: integration tests, basic fs commands
> >       - adl: live unit tests, basic fs commands. See notes below.
> >   - Deployed a pseudo cluster, passed the following sanity tests in
> >   both insecure and SSL mode:
> >       - HDFS: basic dfs, distcp, ACL commands
> >       - KMS and HttpFS: basic tests
> >       - MapReduce wordcount
> >       - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> >   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> >   - HDFS-11132. Allow AccessControlException in contract tests when
> >   getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> >   Dusane
> >   - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> >   runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >  <property>
> > >    <name>fs.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >  </property>
> > >
> > >  <property>
> > >    <name>fs.AbstractFileSystem.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.Adl</value>
> > >  </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <jz...@cloudera.com>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>      With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >>      This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >>      More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >>      New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >>      The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >>      Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Ravi Prakash <ra...@gmail.com>]

Thanks for all the effort Junping!

+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.

I agree none of the issues reported so far are blockers. Looking forward to
another great release.

Thanks
Ravi

On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
>   - Verified checksums and signatures of the tarballs
>   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>   - Cloud connectors:
>       - s3a: integration tests, basic fs commands
>       - adl: live unit tests, basic fs commands. See notes below.
>   - Deployed a pseudo cluster, passed the following sanity tests in
>   both insecure and SSL mode:
>       - HDFS: basic dfs, distcp, ACL commands
>       - KMS and HttpFS: basic tests
>       - MapReduce wordcount
>       - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
>   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>   - HDFS-11132. Allow AccessControlException in contract tests when
>   getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
>   Dusane
>   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>   runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >  <property>
> >    <name>fs.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >  </property>
> >
> >  <property>
> >    <name>fs.AbstractFileSystem.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.Adl</value>
> >  </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >>      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>      This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Ravi Prakash <ra...@gmail.com>]

Thanks for all the effort Junping!

+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.

I agree none of the issues reported so far are blockers. Looking forward to
another great release.

Thanks
Ravi

On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
>   - Verified checksums and signatures of the tarballs
>   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>   - Cloud connectors:
>       - s3a: integration tests, basic fs commands
>       - adl: live unit tests, basic fs commands. See notes below.
>   - Deployed a pseudo cluster, passed the following sanity tests in
>   both insecure and SSL mode:
>       - HDFS: basic dfs, distcp, ACL commands
>       - KMS and HttpFS: basic tests
>       - MapReduce wordcount
>       - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
>   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>   - HDFS-11132. Allow AccessControlException in contract tests when
>   getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
>   Dusane
>   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>   runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >  <property>
> >    <name>fs.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >  </property>
> >
> >  <property>
> >    <name>fs.AbstractFileSystem.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.Adl</value>
> >  </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >>      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>      This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Ravi Prakash <ra...@gmail.com>]

Thanks for all the effort Junping!

+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.

I agree none of the issues reported so far are blockers. Looking forward to
another great release.

Thanks
Ravi

On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
>   - Verified checksums and signatures of the tarballs
>   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>   - Cloud connectors:
>       - s3a: integration tests, basic fs commands
>       - adl: live unit tests, basic fs commands. See notes below.
>   - Deployed a pseudo cluster, passed the following sanity tests in
>   both insecure and SSL mode:
>       - HDFS: basic dfs, distcp, ACL commands
>       - KMS and HttpFS: basic tests
>       - MapReduce wordcount
>       - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
>   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>   - HDFS-11132. Allow AccessControlException in contract tests when
>   getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
>   Dusane
>   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>   runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >  <property>
> >    <name>fs.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >  </property>
> >
> >  <property>
> >    <name>fs.AbstractFileSystem.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.Adl</value>
> >  </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >>      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>      This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Ravi Prakash <ra...@gmail.com>]

Thanks for all the effort Junping!

+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.

I agree none of the issues reported so far are blockers. Looking forward to
another great release.

Thanks
Ravi

On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <jd...@hortonworks.com> wrote:

> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <ks...@yahoo-inc.com>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <eb...@yahoo-inc.com.INVALID> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
>   - Verified checksums and signatures of the tarballs
>   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>   - Cloud connectors:
>       - s3a: integration tests, basic fs commands
>       - adl: live unit tests, basic fs commands. See notes below.
>   - Deployed a pseudo cluster, passed the following sanity tests in
>   both insecure and SSL mode:
>       - HDFS: basic dfs, distcp, ACL commands
>       - KMS and HttpFS: basic tests
>       - MapReduce wordcount
>       - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
>   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>   - HDFS-11132. Allow AccessControlException in contract tests when
>   getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
>   Dusane
>   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>   runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >  <property>
> >    <name>fs.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >  </property>
> >
> >  <property>
> >    <name>fs.AbstractFileSystem.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.Adl</value>
> >  </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <jz...@cloudera.com>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> > >
> > >> Hi all,
> > >>      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>      This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks all for response with verification work and vote!


Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.


We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!


Thanks,


Junping


________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)


+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks all for response with verification work and vote!


Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.


We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!


Thanks,


Junping


________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)


+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks all for response with verification work and vote!


Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.


We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!


Thanks,


Junping


________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)


+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks all for response with verification work and vote!


Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.


We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!


Thanks,


Junping


________________________________
From: Kuhu Shukla <ks...@yahoo-inc.com>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)


+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[zhenke luo <zh...@gmail.com>]

Hello:

I found an issue with HADOOP-10027 that :

SnappyCompressor can’t load native lib properly.

Test TestHadoopnative is failing  in hadoop-apache2 version 0.10 

stacktrace:
Caused by: java.lang.RuntimeException: native snappy library not available: SnappyCompressor has not been loaded.
	at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:72)
	at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:195)
	at com.facebook.presto.hadoop.HadoopNative.loadAllCodecs(HadoopNative.java:71)
	at com.facebook.presto.hadoop.HadoopNative.requireHadoopNative(HadoopNative.java:52)
	... 24 more

After look into the code found that in SnappyCompressor.java 
static {
    if (NativeCodeLoader.isNativeCodeLoaded() &&
        NativeCodeLoader.buildSupportsSnappy()) {
      try {
        initIDs();
        nativeSnappyLoaded = true;
      } catch (Throwable t) {
        LOG.error("failed to load SnappyCompressor", t);
      }
    }
  }
the native method initIDs will complain: no such field clazz.


> On Mar 21, 2017, at 4:32 PM, Poepping, Thomas <po...@amazon.com> wrote:
> 
> 
> 
> On 3/21/17, 3:17 PM, "Kuhu Shukla" <ks...@yahoo-inc.com.INVALID> wrote:
> 
> 
>    +1 (non-binding)
> 
>    - Verified signatures.
>    - Downloaded and built from source tar.gz.
>    - Deployed a pseudo-distributed cluster on Mac Sierra.
>    - Ran example Sleep job successfully.
>    - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
> 
>    Thank you Junping and everyone else who worked on getting this release out.
> 
>    Warm Regards,
>    Kuhu
>    On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
>    +1 (non-binding)
> 
>    - Verified checksums and signatures of all files
>    - Built from source on MacOS Sierra via JDK 1.8.0 u65
>    - Deployed single-node cluster
>    - Successfully ran a few sample jobs
> 
>    Thanks,
> 
>    Eric
> 
>    On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:
> 
> 
> 
>    +1. Thanks for the great effort, Junping!
> 
> 
>      - Verified checksums and signatures of the tarballs
>      - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>      - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>      - Cloud connectors:
>          - s3a: integration tests, basic fs commands
>          - adl: live unit tests, basic fs commands. See notes below.
>      - Deployed a pseudo cluster, passed the following sanity tests in
>      both insecure and SSL mode:
>          - HDFS: basic dfs, distcp, ACL commands
>          - KMS and HttpFS: basic tests
>          - MapReduce wordcount
>          - balancer start/stop
> 
> 
>    Needs the following JIRAs to pass all ADL tests:
> 
>      - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>      - HDFS-11132. Allow AccessControlException in contract tests when
>      getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
>      Dusane
>      - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>      runtime error. (John Zhuge via lei)
> 
> 
>    On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:
> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>> <property>
>>   <name>fs.adl.impl</name>
>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>> 
>> <property>
>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>> 
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>> 
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>> » AccessControl
>> 
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>> » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>> 
>>> Thank you for reporting the issue, John! Does this issue only affect ADL
>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>> existing
>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>> not a
>>> regression and just a new feature get broken.​
>>> 
>>> 
>>> Thanks,
>>> 
>>> 
>>> Junping
>>> ------------------------------
>>> *From:* John Zhuge <jz...@cloudera.com>
>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>> *To:* Junping Du
>>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>> 
>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>> FileSystem for scheme: adl".
>>> 
>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>> earlier.
>>> HADOOP-12666 should not be backported, but some changes are needed:
>>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>> 
>>> I am working on a patch.
>>> 
>>> 
>>> John Zhuge
>>> Software Engineer, Cloudera
>>> 
>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>>> 
>>>> Hi all,
>>>>     With fix of HDFS-11431 get in, I've created a new release candidate
>>>> (RC3) for Apache Hadoop 2.8.0.
>>>> 
>>>>     This is the next minor release to follow up 2.7.0 which has been
>>>> released for more than 1 year. It comprises 2,900+ fixes, improvements,
>> and
>>>> new features. Most of these commits are released for the first time in
>>>> branch-2.
>>>> 
>>>>     More information about the 2.8.0 release plan can be found here:
>>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>>> 
>>>>     New RC is available at: http://home.apache.org/~junpin
>>>> g_du/hadoop-2.8.0-RC3
>>>> 
>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>> 
>>>>     The maven artifacts are available via repository.apache.org at:
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>>> 
>>>>     Please try the release and vote; the vote will run for the usual 5
>>>> days, ending on 03/22/2017 PDT time.
> 
>>>> 
>>>> Thanks,
>>>> 
>>>> Junping
>>>> 
>>> 
>>> 
>> 
> 
> 
> 
>    -- 
>    John
> 
> 
>    ---------------------------------------------------------------------
>    To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>    For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> 
> 

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Kuhu Shukla <ks...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <eb...@yahoo-inc.com.INVALID> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Badger <eb...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Badger <eb...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Badger <eb...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Badger <eb...@yahoo-inc.com.INVALID>]

+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <jo...@gmail.com> wrote:



+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jo...@gmail.com>]

+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jo...@gmail.com>]

+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Zhihai Xu <zh...@gmail.com>]

Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Zhihai Xu <zh...@gmail.com>]

Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Zhihai Xu <zh...@gmail.com>]

Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Zhihai Xu <zh...@gmail.com>]

Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <st...@hortonworks.com>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> >> jdu@hortonworks.com>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <jz...@cloudera.com>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:jd
> >> u@hortonworks.com>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

I'm an =0 I'm afraid

I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of 

HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,

of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.

I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.

so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.

Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those

-Steve




> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> 
> Thanks Junping for working on the new release!
> 
> +1 non-binding
> 
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
> 
> Best,
> Haibo Chen
> 
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>> 
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>>  <property>
>>    <name>fs.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>  </property>
>> 
>>  <property>
>>    <name>fs.AbstractFileSystem.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.Adl</value>
>>  </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> 
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>> 
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>> 
>> I am working on a patch.
>> 
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!

Thanks,

Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!

Thanks,

Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!

Thanks,

Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!

Thanks,

Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here? 
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here? 
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here? 
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here? 
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <aw...@effectivemachines.com>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Allen Wittenauer <aw...@effectivemachines.com>]

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
     Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
     Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
    Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
    Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
    Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
    Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
    Varun Saxena

1 binding +0, from:
    Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​


________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
     Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
     Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
    Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
    Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
    Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
    Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
    Varun Saxena

1 binding +0, from:
    Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​


________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
     Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
     Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
    Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
    Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
    Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
    Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
    Varun Saxena

1 binding +0, from:
    Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​


________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
     Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
     Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
    Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
    Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
    Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
    Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
    Varun Saxena

1 binding +0, from:
    Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​


________________________________
From: Karthik Kambatla <ka...@cloudera.com>
Sent: Wednesday, March 22, 2017 2:10 PM
To: varunsaxena@apache.org
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org<ma...@apache.org> <va...@gmail.com>> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>
> <mailto:
> > jdu@hortonworks.com<ma...@hortonworks.com>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com><mailto:
> jd
> > u@hortonworks.com<ma...@hortonworks.com>>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Karthik Kambatla <ka...@cloudera.com>]

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:

> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >   <property>
> > >     <name>fs.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >   </property>
> > >
> > >   <property>
> > >     <name>fs.AbstractFileSystem.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.Adl</value>
> > >   </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > >   TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > >      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > >      This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > >       More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > >       New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > >       The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > >       Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Karthik Kambatla <ka...@cloudera.com>]

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:

> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >   <property>
> > >     <name>fs.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >   </property>
> > >
> > >   <property>
> > >     <name>fs.AbstractFileSystem.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.Adl</value>
> > >   </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > >   TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > >      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > >      This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > >       More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > >       New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > >       The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > >       Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Karthik Kambatla <ka...@cloudera.com>]

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, varunsaxena@apache.org <
varun.saxena.apache@gmail.com> wrote:

> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >   <property>
> > >     <name>fs.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >   </property>
> > >
> > >   <property>
> > >     <name>fs.AbstractFileSystem.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.Adl</value>
> > >   </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > >   TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> > <mailto:
> > > jdu@hortonworks.com>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <jz...@cloudera.com>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > > mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.apache.org
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jd
> > > u@hortonworks.com>> wrote:
> > > Hi all,
> > >      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > >      This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > >       More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > >       New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > >       The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > >       Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

["varunsaxena@apache.org" <va...@gmail.com>]

Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

["varunsaxena@apache.org" <va...@gmail.com>]

Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

I'm an =0 I'm afraid

I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of 

HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,

of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.

I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.

so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.

Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those

-Steve




> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> 
> Thanks Junping for working on the new release!
> 
> +1 non-binding
> 
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
> 
> Best,
> Haibo Chen
> 
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>> 
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>>  <property>
>>    <name>fs.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>  </property>
>> 
>>  <property>
>>    <name>fs.AbstractFileSystem.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.Adl</value>
>>  </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> 
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>> 
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>> 
>> I am working on a patch.
>> 
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

I'm an =0 I'm afraid

I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of 

HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,

of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.

I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.

so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.

Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those

-Steve




> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> 
> Thanks Junping for working on the new release!
> 
> +1 non-binding
> 
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
> 
> Best,
> Haibo Chen
> 
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>> 
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>>  <property>
>>    <name>fs.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>  </property>
>> 
>>  <property>
>>    <name>fs.AbstractFileSystem.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.Adl</value>
>>  </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> 
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>> 
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>> 
>> I am working on a patch.
>> 
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

["varunsaxena@apache.org" <va...@gmail.com>]

Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

I'm an =0 I'm afraid

I think those extra files need to be pulled which triggers a repackage, at which point there's a couple of 

HADOOP-14205 No FileSystem for scheme: adl
HADOOP-14204 S3A multipart commit failing,

of the two, the ADL FS one is the most significant, though if there was going to be a re-roll, I'd like them in.

I also worry about windows test coverage, as irrespective of whether people are planning to run it on a windows host in production, the hadoop libs get used client side in job submit, and more now in Spark . HADOOP-14201 showed up that (a) my new VM is a mess, so complicating testing and (b) gzip was failing. I Worry about gzip, as that could be a sign of native lib problems.. someone needs to look at it a bit more. However, I don't think it should really be a blocker, given the ASF doesn't officially redistribute windows binaries.

so: some issues with packaging and a couple of patches which need to go in. Test coverage on windows is potentially longer, and I don't want it to hold things up.

Anyhow: =0. If everyone else is happy to ship, then let's get it out the door, as that's the only way that we'll get all the in-the-field bugreps coming back in. We will need to release some new versions in the coming weeks to handle those

-Steve




> On 21 Mar 2017, at 18:43, Haibo Chen <ha...@cloudera.com> wrote:
> 
> Thanks Junping for working on the new release!
> 
> +1 non-binding
> 
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
> 
> Best,
> Haibo Chen
> 
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
>> ?Thanks for update, John. Then we should be OK with fixing this issue in
>> 2.8.1.
>> 
>> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
>> up to blocker in case we could miss this in releasing 2.8.1. :)
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>
>> Sent: Monday, March 20, 2017 10:31 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>> 
>>  <property>
>>    <name>fs.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>  </property>
>> 
>>  <property>
>>    <name>fs.AbstractFileSystem.adl.impl</name>
>>    <value>org.apache.hadoop.fs.adl.Adl</value>
>>  </property>
>> 
>> I have the initial patch ready but hitting these live unit test failures:
>> 
>> Failed tests:
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
>> expected:<1> but was:<10>
>> 
>> Tests in error:
>>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>> 
>> 
>> Stay tuned...
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> 
>> Thank you for reporting the issue, John! Does this issue only affect ADL
>> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
>> regression and just a new feature get broken.?
>> 
>> 
>> Thanks,
>> 
>> 
>> Junping
>> 
>> ________________________________
>> From: John Zhuge <jz...@cloudera.com>>
>> Sent: Monday, March 20, 2017 9:07 AM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>> 
>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>> FileSystem for scheme: adl".
>> 
>> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
>> HADOOP-12666 should not be backported, but some changes are needed:
>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>> 
>> I am working on a patch.
>> 
>> 
>> John Zhuge
>> Software Engineer, Cloudera
>> 
>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
>> u@hortonworks.com>> wrote:
>> Hi all,
>>     With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>> 
>>     This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>> 
>>      More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>> 
>>      New RC is available at: http://home.apache.org/~
>> junping_du/hadoop-2.8.0-RC3
>> 
>>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>> 
>>      The maven artifacts are available via repository.apache.org<http://
>> repository.apache.org> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1057
>> 
>>      Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>> 
>> Thanks,
>> 
>> Junping
>> 
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

["varunsaxena@apache.org" <va...@gmail.com>]

Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <ha...@cloudera.com> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com
> <mailto:
> > jdu@hortonworks.com>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <jz...@cloudera.com>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:
> jd
> > u@hortonworks.com>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Haibo Chen <ha...@cloudera.com>]

Thanks Junping for working on the new release!

+1 non-binding

1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines

Best,
Haibo Chen

On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>   TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Haibo Chen <ha...@cloudera.com>]

Thanks Junping for working on the new release!

+1 non-binding

1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines

Best,
Haibo Chen

On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>   TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Haibo Chen <ha...@cloudera.com>]

Thanks Junping for working on the new release!

+1 non-binding

1) Downloaded the source, verified the checksum
2) Built natively from source, and deployed it to a pseudo-distributed
cluster
3) Ran sleep and teragen job and checked both YARN and JHS web UI
4) Played with yarn + mapreduce command lines

Best,
Haibo Chen

On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> ?Thanks for update, John. Then we should be OK with fixing this issue in
> 2.8.1.
>
> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump
> up to blocker in case we could miss this in releasing 2.8.1. :)
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 10:31 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>   TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
>   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
>
> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.

Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:

  <property>
    <name>fs.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
  </property>

  <property>
    <name>fs.AbstractFileSystem.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.Adl</value>
  </property>

I have the initial patch ready but hitting these live unit test failures:

Failed tests:
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>

Tests in error:
  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl


Stay tuned...

John Zhuge
Software Engineer, Cloudera

On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.

Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:

  <property>
    <name>fs.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
  </property>

  <property>
    <name>fs.AbstractFileSystem.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.Adl</value>
  </property>

I have the initial patch ready but hitting these live unit test failures:

Failed tests:
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>

Tests in error:
  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl


Stay tuned...

John Zhuge
Software Engineer, Cloudera

On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jo...@gmail.com>]

+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.

Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:

  <property>
    <name>fs.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
  </property>

  <property>
    <name>fs.AbstractFileSystem.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.Adl</value>
  </property>

I have the initial patch ready but hitting these live unit test failures:

Failed tests:
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>

Tests in error:
  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl


Stay tuned...

John Zhuge
Software Engineer, Cloudera

On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

?Thanks for update, John. Then we should be OK with fixing this issue in 2.8.1.

Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and bump up to blocker in case we could miss this in releasing 2.8.1. :)


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 10:31 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Yes, it only affects ADL. There is a workaround of adding these 2 properties to core-site.xml:

  <property>
    <name>fs.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
  </property>

  <property>
    <name>fs.AbstractFileSystem.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.Adl</value>
  </property>

I have the initial patch ready but hitting these live unit test failures:

Failed tests:
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257 expected:<1> but was:<10>

Tests in error:
  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl


Stay tuned...

John Zhuge
Software Engineer, Cloudera

On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com>> wrote:

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jo...@gmail.com>]

+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <jz...@cloudera.com> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <jz...@cloudera.com>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> > yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



-- 
John

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jz...@cloudera.com>]

Yes, it only affects ADL. There is a workaround of adding these 2
properties to core-site.xml:

  <property>
    <name>fs.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
  </property>

  <property>
    <name>fs.AbstractFileSystem.adl.impl</name>
    <value>org.apache.hadoop.fs.adl.Adl</value>
  </property>

I have the initial patch ready but hitting these live unit test failures:

Failed tests:

TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:257
expected:<1> but was:<10>

Tests in error:

TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:254
» AccessControl

TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testMkdirsFailsForSubdirectoryOfExistingFile:190
» AccessControl


Stay tuned...

John Zhuge
Software Engineer, Cloudera

On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:

> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.​
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* John Zhuge <jz...@cloudera.com>
> *Sent:* Monday, March 20, 2017 9:07 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi all,
>>      With fix of HDFS-11431 get in, I've created a new release candidate
>> (RC3) for Apache Hadoop 2.8.0.
>>
>>      This is the next minor release to follow up 2.7.0 which has been
>> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
>> new features. Most of these commits are released for the first time in
>> branch-2.
>>
>>       More information about the 2.8.0 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>>       New RC is available at: http://home.apache.org/~junpin
>> g_du/hadoop-2.8.0-RC3
>>
>>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>
>>       The maven artifacts are available via repository.apache.org at:
>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>
>>       Please try the release and vote; the vote will run for the usual 5
>> days, ending on 03/22/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>>
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wangda Tan <wh...@gmail.com>]

Thanks Junping for doing this.

*+1 (Binding),*

Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.

Thanks,
Wangda


On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:

> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Wangda Tan <wh...@gmail.com>]

Thanks Junping for doing this.

*+1 (Binding),*

Built from source code, deployed a single node cluster. Enabled node labels
and tried to run sample jobs. Haven't seen any issue so far.

Thanks,
Wangda


On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <jd...@hortonworks.com> wrote:

> Thank you for reporting the issue, John! Does this issue only affect ADL
> (Azure Data Lake) which is a new feature for 2.8 rather than other existing
> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a
> regression and just a new feature get broken.?
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: John Zhuge <jz...@cloudera.com>
> Sent: Monday, March 20, 2017 9:07 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> FileSystem for scheme: adl".
>
> The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
> HADOOP-12666 should not be backported, but some changes are needed:
> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>
> I am working on a patch.
>
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org<http://
> repository.apache.org> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Thank you for reporting the issue, John! Does this issue only affect ADL (Azure Data Lake) which is a new feature for 2.8 rather than other existing FS? If so, I think we can leave the fix to 2.8.1 to fix given this is not a regression and just a new feature get broken.?


Thanks,


Junping

________________________________
From: John Zhuge <jz...@cloudera.com>
Sent: Monday, March 20, 2017 9:07 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier. HADOOP-12666 should not be backported, but some changes are needed: property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jz...@cloudera.com>]

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
HADOOP-12666 should not be backported, but some changes are needed:
property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Akira Ajisaka <aa...@apache.org>]

Thanks Junping!

+1 (binding)

* Verified signatures and checksums
* Built Hive 2.1.0 and Tez 0.8.5 with Hadoop 2.8.0 pom
* Deployed a single node cluster and ran some Hive on Tez queries 
successfully
* The document looks good.

I found a trivial issue in the doc. It does not block the release.
https://issues.apache.org/jira/browse/HADOOP-14208

Regards,
Akira


On 2017/03/17 18:18, Junping Du wrote:
> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>       More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:


Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow



1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked

I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.

Can other people with windows development setups do a run of this?

I've not done any other testing, Windows was enough for a weekend afternoon


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
   With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

   This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

    More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

    New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

    The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

    The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

    Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:


Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow



1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked

I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.

Can other people with windows development setups do a run of this?

I've not done any other testing, Windows was enough for a weekend afternoon


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
   With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

   This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

    More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

    New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

    The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

    The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

    Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:


Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow



1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked

I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.

Can other people with windows development setups do a run of this?

I've not done any other testing, Windows was enough for a weekend afternoon


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
   With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

   This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

    More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

    New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

    The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

    The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

    Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

On 19 Mar 2017, at 15:23, Steve Loughran <st...@hortonworks.com>> wrote:


Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow



1. None of the kerberos auth tests like me
2. many of the hadoop-common failures were test side, with some more diagnostics, fixed
https://issues.apache.org/jira/browse/HADOOP-14201
3. All the hadoop-aws integration tests worked

I had problems with: gzip code and some of the IPC tests, those being so foundational that it helps convince me that the network is somehow at fault, like the VM isn't coming up right.

Can other people with windows development setups do a run of this?

I've not done any other testing, Windows was enough for a weekend afternoon


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
   With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

   This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

    More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

    New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

    The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

    The maven artifacts are available via repository.apache.org<http://repository.apache.org><http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

    Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

     More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

     New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

     The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

     The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

     Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Mingliang Liu <li...@gmail.com>]

Thanks Junping for doing this.

+1 (non-binding)

0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs

L

> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
> 
>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
> 
> Thanks,
> 
> Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

     More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

     New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

     The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

     The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

     Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Daniel Templeton <da...@cloudera.com>]

Thanks for the new RC, Junping.  I built from source and tried it out on 
a 2-node cluster with HA enabled.  I ran a pi job and some streaming 
jobs.  I tested that localization and failover work correctly, and I 
played a little with the YARN and HDFS web UIs.

I did encounter an old friend of mine, which is that if you submit a 
streaming job with input that is only 1 block, you will nonetheless get 
2 mappers that both process the same split. What's new this time is that 
the second mapper was consistently failing on certain input sizes.  I 
(re)verified that the issue also exists is 2.7.3, so it's not a 
regression.  I'm pretty sure it's been there since at least 2.6.0.  I 
filed MAPREDUCE-6864 for it.

Given that my issue was not a regression, I'm +1 on the RC.

Daniel

On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[John Zhuge <jz...@cloudera.com>]

Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
FileSystem for scheme: adl".

The issue were caused by backporting HADOOP-13037 to branch-2 and earlier.
HADOOP-12666 should not be backported, but some changes are needed:
property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.

I am working on a patch.


John Zhuge
Software Engineer, Cloudera

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Daniel Templeton <da...@cloudera.com>]

Thanks for the new RC, Junping.  I built from source and tried it out on 
a 2-node cluster with HA enabled.  I ran a pi job and some streaming 
jobs.  I tested that localization and failover work correctly, and I 
played a little with the YARN and HDFS web UIs.

I did encounter an old friend of mine, which is that if you submit a 
streaming job with input that is only 1 block, you will nonetheless get 
2 mappers that both process the same split. What's new this time is that 
the second mapper was consistently failing on certain input sizes.  I 
(re)verified that the issue also exists is 2.7.3, so it's not a 
regression.  I'm pretty sure it's been there since at least 2.6.0.  I 
filed MAPREDUCE-6864 for it.

Given that my issue was not a regression, I'm +1 on the RC.

Daniel

On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

RE: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Brahma Reddy Battula <br...@huawei.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

verified the following:
1. Deployed HA cluster with three Nodes.
2. Verified the basic shell operations
3. Ran wordount and pi jobs

Regards
Brahma Reddy Battula

-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com] 
Sent: 17 March 2017 17:18
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Jason Lowe <jl...@yahoo-inc.com.INVALID>]

+1 (binding)
- Verfied signatures and digests- Performed a native build from the release tag- Deployed to a single node cluster- Ran some sample jobs
Jason
 

    On Friday, March 17, 2017 4:18 AM, Junping Du <jd...@hortonworks.com> wrote:
 

 Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

   

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

     More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

     New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

     The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

     The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

     Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Miklos Szegedi <mi...@cloudera.com>]

Hi Junping,

Thank you for working on this.

+1 (Non-Binding)

I verified the following:
1. Deployed on a 3 node cluster with 2 node managers.
2. Configured linux container executor
3. Configured fair scheduler
4. Ran Pi job and verified the results
5. Ran multiple Yarn applications and verified the results

Thank you,
Miklos Szegedi

On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Steve Loughran <st...@hortonworks.com>]

Thanks, testing in progress

here's the windows native binaries to go with it, all signed

https://github.com/steveloughran/winutils/releases/tag/hadoop-2.8.0-RC3

The gnu shell commands which came with git worked well here, and the build went through fast, nice to see

Test results to follow


On 17 Mar 2017, at 09:18, Junping Du <jd...@hortonworks.com>> wrote:

Hi all,
    With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

    This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

     More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

     New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

     The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

     The maven artifacts are available via repository.apache.org<http://repository.apache.org> at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

     Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Sunil Govind <su...@gmail.com>]

+1 (non-binding). Thanks Junping for the effort.

I have used release package and verified below cases
- Ran MR sleep job and wordcount successfully in where nodes are configured
with labels.
- Verified application priority feature and I could see high priority apps
are getting resource over lower priority apps when configured
- Verified RM web UI pages and looks fine (priority could be seen)
- Intra-queue preemption related to app priority also seems fine

Thanks
Sunil


On Fri, Mar 17, 2017 at 2:48 PM Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With fix of HDFS-11431 get in, I've created a new release candidate
> (RC3) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,900+ fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       New RC is available at:
> http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Daniel Templeton <da...@cloudera.com>]

Thanks for the new RC, Junping.  I built from source and tried it out on 
a 2-node cluster with HA enabled.  I ran a pi job and some streaming 
jobs.  I tested that localization and failover work correctly, and I 
played a little with the YARN and HDFS web UIs.

I did encounter an old friend of mine, which is that if you submit a 
streaming job with input that is only 1 block, you will nonetheless get 
2 mappers that both process the same split. What's new this time is that 
the second mapper was consistently failing on certain input sizes.  I 
(re)verified that the issue also exists is 2.7.3, so it's not a 
regression.  I'm pretty sure it's been there since at least 2.6.0.  I 
filed MAPREDUCE-6864 for it.

Given that my issue was not a regression, I'm +1 on the RC.

Daniel

On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Masatake Iwasaki <iw...@oss.nttdata.co.jp>]

Thanks for putting this up, Junping.

+1(non-binding)

* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller 
context enabled.

Masatake Iwasaki

On 3/17/17 18:18, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Payne <er...@yahoo.com.INVALID>]

+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI




________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org> 
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)



Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.


Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Eric Payne <er...@yahoo.com.INVALID>]

+1
Thanks, Junping, for your efforts to get this release out.
I downloaded and built the source, and i did the following manual testing on a 2-node pseudo cluster:
- Streaming job
- Inter-queue (cross-queue) preemption--verified that only the expected amount of preemptions occured.
- Inter-queue (in-queue) preemption with higher priority apps preempting lower ones.
- Limited node label testing.
- Yarn distributed shell, both with and without keeping containers across AM restart.
- Killing apps from Application UI




________________________________
From: Junping Du <jd...@hortonworks.com>
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>; "hdfs-dev@hadoop.apache.org" <hd...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org> 
Sent: Friday, March 17, 2017 4:18 AM
Subject: [VOTE] Release Apache Hadoop 2.8.0 (RC3)



Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.


Thanks,

Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Mingliang Liu <li...@gmail.com>]

Thanks Junping for doing this.

+1 (non-binding)

0. Download the src tar.gz file; checked the MD5 checksum
1. Build Hadoop from source successfully
2. Deploy a single node cluster and start the cluster successfully
3. Operate the HDFS from command line: ls, put, distcp, dfsadmin etc
4. Run hadoop mapreduce examples: grep
5. Operate AWS S3 using S3A schema from commandline: ls, cat, distcp
6. Check the HDFS service logs

L

> On Mar 17, 2017, at 2:18 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
> 
>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
> 
> Thanks,
> 
> Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Daniel Templeton <da...@cloudera.com>]

Thanks for the new RC, Junping.  I built from source and tried it out on 
a 2-node cluster with HA enabled.  I ran a pi job and some streaming 
jobs.  I tested that localization and failover work correctly, and I 
played a little with the YARN and HDFS web UIs.

I did encounter an old friend of mine, which is that if you submit a 
streaming job with input that is only 1 block, you will nonetheless get 
2 mappers that both process the same split. What's new this time is that 
the second mapper was consistently failing on certain input sizes.  I 
(re)verified that the issue also exists is 2.7.3, so it's not a 
regression.  I'm pretty sure it's been there since at least 2.6.0.  I 
filed MAPREDUCE-6864 for it.

Given that my issue was not a regression, I'm +1 on the RC.

Daniel

On 3/17/17 2:18 AM, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Masatake Iwasaki <iw...@oss.nttdata.co.jp>]

Thanks for putting this up, Junping.

+1(non-binding)

* verified signature and mds.
* built from source on centos 7 and openjdk 8 with native profile.
* built docs and skimmed the contents.
* deployed 3 nodes ha cluster and run some sample jobs with caller 
context enabled.

Masatake Iwasaki

On 3/17/17 18:18, Junping Du wrote:
> Hi all,
>       With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.
>
>       This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>        More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>        New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3
>
>        The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>
>        The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057
>
>        Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.
>
> Thanks,
>
> Junping
>


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

[VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects
but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With several important fixes get merged last week, I've created a new
> release candidate (RC2) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,919 fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       Please note that RC0 and RC1 are not voted public because
> significant issues are found just after RC tag getting published.
>
>       The RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC2
>
>       The RC tag in git is: release-2.8.0-RC2
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1056
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping
>

[VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

Hi Junping,

Noticed this possible blocker float by my inbox today. It had an affects
but no target version set:

https://issues.apache.org/jira/browse/HDFS-11431

Thoughts? Seems like the hadoop-hdfs-client artifact doesn't work right now.

Best,
Andrew


On Tue, Mar 14, 2017 at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:

> Hi all,
>      With several important fixes get merged last week, I've created a new
> release candidate (RC2) for Apache Hadoop 2.8.0.
>
>      This is the next minor release to follow up 2.7.0 which has been
> released for more than 1 year. It comprises 2,919 fixes, improvements, and
> new features. Most of these commits are released for the first time in
> branch-2.
>
>       More information about the 2.8.0 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>       Please note that RC0 and RC1 are not voted public because
> significant issues are found just after RC tag getting published.
>
>       The RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.0-RC2
>
>       The RC tag in git is: release-2.8.0-RC2
>
>       The maven artifacts are available via repository.apache.org at:
> https://repository.apache.org/content/repositories/orgapachehadoop-1056
>
>       Please try the release and vote; the vote will run for the usual 5
> days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping
>

[VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Vinod Kumar Vavilapalli <vi...@apache.org>]

Thanks for taking the mantle from me on 2.8.0 and some persistent work getting 2.8.0 out the door, Junping!

Apologies for bringing this up late, but I’d like to add one comment.

We should repeat what we did for 2.7.0 and In line with our experience there, we should annotate this release as not ready for production use. See the releases page - http://hadoop.apache.org/releases.html#25+August%2C+2016%3A+Release+2.7.3+available for our messaging on 2.7.0.

The expectation is that more downstream projects pick up the bits, iron out any incompatibilities we might have missed, and production users then pick up a solid 2.8.1.

Thanks
+Vinod

> On Mar 14, 2017, at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping


---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

[VOTE] Release Apache Hadoop 2.8.0 (RC3)

[Junping Du <jd...@hortonworks.com>]

Hi all,
     With fix of HDFS-11431 get in, I've created a new release candidate (RC3) for Apache Hadoop 2.8.0.

     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,900+ fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.

      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

      New RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC3

      The RC tag in git is: release-2.8.0-RC3, and the latest commit id is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009

      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1057

      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/22/2017 PDT time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

On Mon, Mar 20, 2017 at 5:30 AM, Steve Loughran <st...@hortonworks.com>
wrote:

>
> On 15 Mar 2017, at 21:06, Eric Badger <eb...@yahoo-inc.com> wrote:
>
> Verified signatures
>  - Minor note: Junping, I had a hard time finding your key. I grabbed the
> keys for hadoop from
> http://home.apache.org/keys/group/hadoop.asc and you had a key there, but
> it wasn't the one that you signed this commit with. Then with some help
> from Jason I found the correct key at
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS. So it
> would be nice if those were in sync.
> Compiled from source
> Deployed pseudo-distributed cluster
> Ran some sample MR jobs
>
>
>
> we need to do more key signing; the stuff in the various KEYS files have
> aged
>
> Alll ASF Committers can publish their ASF keys:
>
> https://people.apache.org/keys/committer/
>
> which you can retrieve on a committer-by-committer basis :
>
> junping https://people.apache.org/keys/committer/junping_du.asc
> me: https://people.apache.org/keys/committer/stevel.asc
>
> Committers should log in to https://id.apache.org/ and set them.
>
> Maybe that committer page should just be declared as the reference place
> to find keys; It bootstraps off the ASF HTTPS certificate for trusted D/L,
> and relies on login credentials being kept secure. But if not, well, people
> can publish code under your login, so signing is the least concern.
>
>
Hi Steve,

I said this in a previous email in this thread, but per INFRA we're not to
rely on the keys set on id.apache.org for release verification. Keys need
to be added to the dist KEYS file.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 21:06, Eric Badger <eb...@yahoo-inc.com> wrote:
> 
> Verified signatures
>  - Minor note: Junping, I had a hard time finding your key. I grabbed the keys for hadoop from
> http://home.apache.org/keys/group/hadoop.asc <http://home.apache.org/keys/group/hadoop.asc> and you had a key there, but it wasn't the one that you signed this commit with. Then with some help from Jason I found the correct key at
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>. So it would be nice if those were in sync.
> Compiled from source
> Deployed pseudo-distributed cluster
> Ran some sample MR jobs


we need to do more key signing; the stuff in the various KEYS files have aged

Alll ASF Committers can publish their ASF keys:

https://people.apache.org/keys/committer/ <https://people.apache.org/keys/committer/>

which you can retrieve on a committer-by-committer basis :

junping https://people.apache.org/keys/committer/junping_du.asc <https://people.apache.org/keys/committer/junping_du.asc>
me: https://people.apache.org/keys/committer/stevel.asc <https://people.apache.org/keys/committer/stevel.asc>

Committers should log in to https://id.apache.org/ <https://id.apache.org/> and set them.

Maybe that committer page should just be declared as the reference place to find keys; It bootstraps off the ASF HTTPS certificate for trusted D/L, and relies on login credentials being kept secure. But if not, well, people can publish code under your login, so signing is the least concern.

-Steve

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Andrew Wang <an...@cloudera.com>]

On Wed, Mar 15, 2017 at 5:42 PM, Junping Du <jd...@hortonworks.com> wrote:

> Hi Eric,
>      Thanks for your verification work! About your question on RM's key,
> we actually mentioned we were using  https://dist.apache.org/repos/
> dist/release/hadoop/common/KEYS in our hadoop wiki page:
> https://wiki.apache.org/hadoop/HowToRelease. Also, for hadoop user, our
> release page (http://hadoop.apache.org/releases.html) points key file
> location to the same place. So for developers and users in hadoop
> community, I hope this is not confusing too much.
>      However, from my offline check with Owen, it sounds like
> http://home.apache.org/keys/group/hadoop.asc is something tradition for
> apache projects and convenient for usage. I already updated related key to
> my apache id which should sync to there automatically. We'd better document
> it also in our hadoop wiki page.
>
> I actually asked INFRA about this when I was adding my key, a little more
backstory:

We used to have a README in dist saying to add your key on id.apache.org,
then to export the hadoop group's keys to generate dist's KEYS file.

INFRA told me this is a Bad Thing, since the KEYS file should be append
only. This way, users can still verify a release even if an RM leaves the
hadoop group or changes their key on id.apache.org.

So, I deleted the old README instructions. The dist KEYS file is the
canonical (and only) place to look for an RM's keys. Based on Junping's
examination, it sounds like our docs to reflect this. I'd rather not
complicate matters by also discussing the hadoop group's keys.

Best,
Andrew

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

Hi Eric,
     Thanks for your verification work! About your question on RM's key, we actually mentioned we were using  https://dist.apache.org/repos/dist/release/hadoop/common/KEYS in our hadoop wiki page: https://wiki.apache.org/hadoop/HowToRelease. Also, for hadoop user, our release page (http://hadoop.apache.org/releases.html) points key file location to the same place. So for developers and users in hadoop community, I hope this is not confusing too much.
     However, from my offline check with Owen, it sounds like http://home.apache.org/keys/group/hadoop.asc is something tradition for apache projects and convenient for usage. I already updated related key to my apache id which should sync to there automatically. We'd better document it also in our hadoop wiki page.  

Thanks,

Junping
________________________________________
From: Eric Badger <eb...@yahoo-inc.com>
Sent: Wednesday, March 15, 2017 2:06 PM
To: Junping Du; Steve Loughran
Cc: common-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

All on MacOS Sierra

Verified signatures
  - Minor note: Junping, I had a hard time finding your key. I grabbed the keys for hadoop from
http://home.apache.org/keys/group/hadoop.asc and you had a key there, but it wasn't the one that you signed this commit with. Then with some help from Jason I found the correct key at
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS. So it would be nice if those were in sync.
Compiled from source
Deployed pseudo-distributed cluster
Ran some sample MR jobs

+1 (non-binding)

Thanks,

Eric


On Wednesday, March 15, 2017 2:58 PM, Junping Du <jd...@hortonworks.com> wrote:



The latest commit on RC2 is: e51312e8e106efb2ebd4844eecacb51026fac8b7.
btw, I think tags are immutable. Isn't it?

Thanks,

Junping
________________________________________

From: Steve Loughran
Sent: Wednesday, March 15, 2017 12:30 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 14 Mar 2017, at 08:41, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
>
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
>
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
>
>      The RC tag in git is: release-2.8.0-RC2

given tags are so easy to move, we need to be relying on one or more of:
-the commit ID,
-the tag being signed

Junping: what is the commit Id for the release?

>
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
>

thanks, I'll play with these downstream, as well as checking out and trying to build on windows

>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Eric Badger <eb...@yahoo-inc.com.INVALID>]

All on MacOS Sierra

Verified signatures
  - Minor note: Junping, I had a hard time finding your key. I grabbed the keys for hadoop from 
http://home.apache.org/keys/group/hadoop.asc and you had a key there, but it wasn't the one that you signed this commit with. Then with some help from Jason I found the correct key at 
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS. So it would be nice if those were in sync. 
Compiled from source
Deployed pseudo-distributed cluster
Ran some sample MR jobs

+1 (non-binding)

Thanks,

Eric


On Wednesday, March 15, 2017 2:58 PM, Junping Du <jd...@hortonworks.com> wrote:



The latest commit on RC2 is: e51312e8e106efb2ebd4844eecacb51026fac8b7.
btw, I think tags are immutable. Isn't it?

Thanks,

Junping
________________________________________

From: Steve Loughran
Sent: Wednesday, March 15, 2017 12:30 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 14 Mar 2017, at 08:41, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
>
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
>
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
>
>      The RC tag in git is: release-2.8.0-RC2

given tags are so easy to move, we need to be relying on one or more of:
-the commit ID,
-the tag being signed

Junping: what is the commit Id for the release?

>
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
>

thanks, I'll play with these downstream, as well as checking out and trying to build on windows

>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 15 Mar 2017, at 23:04, Josh Elser <el...@apache.org> wrote:
> 
> A tag is immutable, but you (or someone else) could remove the tag you pushed and re-push a new one. That's why the commit id is important -- it ensures that everyone else knows the exact commit being voted on.
> 

There's tag signing too, "git tag --sign". We can/should use that for authenticating tags, saying that "the release is tag 2.8.x signed by me"

> Junping Du wrote:
>> The latest commit on RC2 is: e51312e8e106efb2ebd4844eecacb51026fac8b7.
>> btw, I think tags are immutable. Isn't it?
>> 
>> Thanks,
>> 
>> Junping
>> ________________________________________
>> From: Steve Loughran
>> Sent: Wednesday, March 15, 2017 12:30 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org
>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
>> 
>>> On 14 Mar 2017, at 08:41, Junping Du<jd...@hortonworks.com>  wrote:
>>> 
>>> Hi all,
>>>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
>>> 
>>>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>> 
>>>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>> 
>>>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
>>> 
>>>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
>>> 
>>>      The RC tag in git is: release-2.8.0-RC2
>> 
>> given tags are so easy to move, we need to be relying on one or more of:
>> -the commit ID,
>> -the tag being signed
>> 
>> Junping: what is the commit Id for the release?
>> 
>>>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
>>> 
>> 
>> thanks, I'll play with these downstream, as well as checking out and trying to build on windows
>> 
>>>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
>>> 
>>> Thanks,
>>> 
>>> Junping
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Josh Elser <el...@apache.org>]

A tag is immutable, but you (or someone else) could remove the tag you 
pushed and re-push a new one. That's why the commit id is important -- 
it ensures that everyone else knows the exact commit being voted on.

Junping Du wrote:
> The latest commit on RC2 is: e51312e8e106efb2ebd4844eecacb51026fac8b7.
> btw, I think tags are immutable. Isn't it?
>
> Thanks,
>
> Junping
> ________________________________________
> From: Steve Loughran
> Sent: Wednesday, March 15, 2017 12:30 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)
>
>> On 14 Mar 2017, at 08:41, Junping Du<jd...@hortonworks.com>  wrote:
>>
>> Hi all,
>>      With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
>>
>>      This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>>
>>       More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>>       Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
>>
>>       The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
>>
>>       The RC tag in git is: release-2.8.0-RC2
>
> given tags are so easy to move, we need to be relying on one or more of:
> -the commit ID,
> -the tag being signed
>
> Junping: what is the commit Id for the release?
>
>>       The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
>>
>
> thanks, I'll play with these downstream, as well as checking out and trying to build on windows
>
>>       Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
>>
>> Thanks,
>>
>> Junping
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Junping Du <jd...@hortonworks.com>]

The latest commit on RC2 is: e51312e8e106efb2ebd4844eecacb51026fac8b7.
btw, I think tags are immutable. Isn't it?

Thanks,

Junping
________________________________________
From: Steve Loughran
Sent: Wednesday, March 15, 2017 12:30 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

> On 14 Mar 2017, at 08:41, Junping Du <jd...@hortonworks.com> wrote:
>
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
>
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
>
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
>
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
>
>      The RC tag in git is: release-2.8.0-RC2

given tags are so easy to move, we need to be relying on one or more of:
-the commit ID,
-the tag being signed

Junping: what is the commit Id for the release?

>
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
>

thanks, I'll play with these downstream, as well as checking out and trying to build on windows

>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
>
> Thanks,
>
> Junping

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Steve Loughran <st...@hortonworks.com>]

> On 14 Mar 2017, at 08:41, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2

given tags are so easy to move, we need to be relying on one or more of: 
-the commit ID, 
-the tag being signed

Junping: what is the commit Id for the release?

> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 

thanks, I'll play with these downstream, as well as checking out and trying to build on windows

>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Vinod Kumar Vavilapalli <vi...@apache.org>]

Thanks for taking the mantle from me on 2.8.0 and some persistent work getting 2.8.0 out the door, Junping!

Apologies for bringing this up late, but I’d like to add one comment.

We should repeat what we did for 2.7.0 and In line with our experience there, we should annotate this release as not ready for production use. See the releases page - http://hadoop.apache.org/releases.html#25+August%2C+2016%3A+Release+2.7.3+available for our messaging on 2.7.0.

The expectation is that more downstream projects pick up the bits, iron out any incompatibilities we might have missed, and production users then pick up a solid 2.8.1.

Thanks
+Vinod

> On Mar 14, 2017, at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC2)

[Vinod Kumar Vavilapalli <vi...@apache.org>]

Thanks for taking the mantle from me on 2.8.0 and some persistent work getting 2.8.0 out the door, Junping!

Apologies for bringing this up late, but I’d like to add one comment.

We should repeat what we did for 2.7.0 and In line with our experience there, we should annotate this release as not ready for production use. See the releases page - http://hadoop.apache.org/releases.html#25+August%2C+2016%3A+Release+2.7.3+available for our messaging on 2.7.0.

The expectation is that more downstream projects pick up the bits, iron out any incompatibilities we might have missed, and production users then pick up a solid 2.8.1.

Thanks
+Vinod

> On Mar 14, 2017, at 1:41 AM, Junping Du <jd...@hortonworks.com> wrote:
> 
> Hi all,
>     With several important fixes get merged last week, I've created a new release candidate (RC2) for Apache Hadoop 2.8.0.
> 
>     This is the next minor release to follow up 2.7.0 which has been released for more than 1 year. It comprises 2,919 fixes, improvements, and new features. Most of these commits are released for the first time in branch-2.
> 
>      More information about the 2.8.0 release plan can be found here: https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> 
>      Please note that RC0 and RC1 are not voted public because significant issues are found just after RC tag getting published.
> 
>      The RC is available at: http://home.apache.org/~junping_du/hadoop-2.8.0-RC2
> 
>      The RC tag in git is: release-2.8.0-RC2
> 
>      The maven artifacts are available via repository.apache.org at: https://repository.apache.org/content/repositories/orgapachehadoop-1056
> 
>      Please try the release and vote; the vote will run for the usual 5 days, ending on 03/20/2017 PDT time.
> 
> Thanks,
> 
> Junping


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org