You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by slfan1989 <sl...@apache.org> on 2024/01/11 05:15:37 UTC
[VOTE] Release Apache Hadoop 3.4.0 RC0
Hello all,
We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
hadoop 3.4.0-RC version.
The RC is available at:
https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
Maven artifacts is built by x86 machine and are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1391/
My public key:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
Changelog:
https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
Release notes:
https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
This is a relatively big release (by Hadoop standard) containing about 2852
commits.
Please give it a try, this RC vote will run for 7 days.
Feature highlights:
DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
----------------------------------------
[HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
FsDatasetImpl datasetLock
When there are many namespaces in a large cluster.
YARN Federation improvements
----------------------------------------
[YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
improvements, including the following:
1. YARN Router now boasts a full implementation of all relevant interfaces
including the ApplicationClientProtocol,
ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
2. Enhanced support for Application cleanup and automatic offline
mechanisms for SubCluster are now facilitated by the YARN Router.
3. Code optimization for Router and AMRMProxy was undertaken, coupled with
improvements to previously pending functionalities.
4. Audit logs and Metrics for Router received upgrades.
5. A boost in cluster security features was achieved, with the inclusion of
Kerberos support.
6. The page function of the router has been enhanced.
Upgrade AWS SDK to V2
----------------------------------------
[HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
The S3A connector now uses the V2 AWS SDK. This is a significant change at
the source code level.
Any applications using the internal extension/override points in the
filesystem connector are likely to break.
Consult the document aws\_sdk\_upgrade for the full details.
hadoop-thirdparty will also provide the new RC0 soon.
Best Regards,
Shilun Fan.
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for driving this release, Shilun Fan.
The top page of site documentation (in hadoop-3.4.0-RC0-site.tar.gz)
looks the same as 3.3.5.
While the index.md.vm is updated in branch-3.4.0[1], it seems not to be
reflected.
release-3.4.0-RC0 tag should be pushed to make checking easier.
In addition, the description about new features of previous release
should be removed from the index.md.vm.
[1]
https://github.com/apache/hadoop/blob/branch-3.4.0/hadoop-project/src/site/markdown/index.md.vm
Masatake Iwasaki
On 2024/01/11 14:15, slfan1989 wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for driving this release, Shilun Fan.
The top page of site documentation (in hadoop-3.4.0-RC0-site.tar.gz)
looks the same as 3.3.5.
While the index.md.vm is updated in branch-3.4.0[1], it seems not to be
reflected.
release-3.4.0-RC0 tag should be pushed to make checking easier.
In addition, the description about new features of previous release
should be removed from the index.md.vm.
[1]
https://github.com/apache/hadoop/blob/branch-3.4.0/hadoop-project/src/site/markdown/index.md.vm
Masatake Iwasaki
On 2024/01/11 14:15, slfan1989 wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for driving this release, Shilun Fan.
The top page of site documentation (in hadoop-3.4.0-RC0-site.tar.gz)
looks the same as 3.3.5.
While the index.md.vm is updated in branch-3.4.0[1], it seems not to be
reflected.
release-3.4.0-RC0 tag should be pushed to make checking easier.
In addition, the description about new features of previous release
should be removed from the index.md.vm.
[1]
https://github.com/apache/hadoop/blob/branch-3.4.0/hadoop-project/src/site/markdown/index.md.vm
Masatake Iwasaki
On 2024/01/11 14:15, slfan1989 wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
wonderful! I'll be testing over the weekend
Meanwhile, new changes I'm putting in to trunk are tagged as fixed in 3.5.0
-correct?
steve
On Thu, 11 Jan 2024 at 05:15, slfan1989 <sl...@apache.org> wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for driving this release, Shilun Fan.
The top page of site documentation (in hadoop-3.4.0-RC0-site.tar.gz)
looks the same as 3.3.5.
While the index.md.vm is updated in branch-3.4.0[1], it seems not to be
reflected.
release-3.4.0-RC0 tag should be pushed to make checking easier.
In addition, the description about new features of previous release
should be removed from the index.md.vm.
[1]
https://github.com/apache/hadoop/blob/branch-3.4.0/hadoop-project/src/site/markdown/index.md.vm
Masatake Iwasaki
On 2024/01/11 14:15, slfan1989 wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
wonderful! I'll be testing over the weekend
Meanwhile, new changes I'm putting in to trunk are tagged as fixed in 3.5.0
-correct?
steve
On Thu, 11 Jan 2024 at 05:15, slfan1989 <sl...@apache.org> wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
wonderful! I'll be testing over the weekend
Meanwhile, new changes I'm putting in to trunk are tagged as fixed in 3.5.0
-correct?
steve
On Thu, 11 Jan 2024 at 05:15, slfan1989 <sl...@apache.org> wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>
Re: [VOTE] Release Apache Hadoop 3.4.0 RC0
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
wonderful! I'll be testing over the weekend
Meanwhile, new changes I'm putting in to trunk are tagged as fixed in 3.5.0
-correct?
steve
On Thu, 11 Jan 2024 at 05:15, slfan1989 <sl...@apache.org> wrote:
> Hello all,
>
> We plan to release hadoop 3.4.0 based on hadoop trunk, which is the first
> hadoop 3.4.0-RC version.
>
> The RC is available at:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/ (for amd64)
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-arm64/ (for arm64)
>
> Maven artifacts is built by x86 machine and are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1391/
>
> My public key:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Changelog:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/CHANGELOG.md
>
> Release notes:
> https://home.apache.org/~slfan1989/hadoop-3.4.0-RC0-amd64/RELEASENOTES.md
>
> This is a relatively big release (by Hadoop standard) containing about 2852
> commits.
>
> Please give it a try, this RC vote will run for 7 days.
>
> Feature highlights:
>
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool
> ----------------------------------------
> [HDFS-15180](https://issues.apache.org/jira/browse/HDFS-15180) Split
> FsDatasetImpl datasetLock via blockpool to solve the issue of heavy
> FsDatasetImpl datasetLock
> When there are many namespaces in a large cluster.
>
> YARN Federation improvements
> ----------------------------------------
> [YARN-5597](https://issues.apache.org/jira/browse/YARN-5597) brings many
> improvements, including the following:
>
> 1. YARN Router now boasts a full implementation of all relevant interfaces
> including the ApplicationClientProtocol,
> ResourceManagerAdministrationProtocol, and RMWebServiceProtocol.
> 2. Enhanced support for Application cleanup and automatic offline
> mechanisms for SubCluster are now facilitated by the YARN Router.
> 3. Code optimization for Router and AMRMProxy was undertaken, coupled with
> improvements to previously pending functionalities.
> 4. Audit logs and Metrics for Router received upgrades.
> 5. A boost in cluster security features was achieved, with the inclusion of
> Kerberos support.
> 6. The page function of the router has been enhanced.
>
> Upgrade AWS SDK to V2
> ----------------------------------------
> [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073)
> The S3A connector now uses the V2 AWS SDK. This is a significant change at
> the source code level.
> Any applications using the internal extension/override points in the
> filesystem connector are likely to break.
> Consult the document aws\_sdk\_upgrade for the full details.
>
> hadoop-thirdparty will also provide the new RC0 soon.
>
> Best Regards,
> Shilun Fan.
>