You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Ajay Kumar <aj...@hortonworks.com> on 2019/04/08 22:23:53 UTC
[VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Hi all,
We have created the first release candidate (RC0) for Apache Hadoop Ozone 0.4.0-alpha.
This release contains security payload for Ozone. Below are some important features in it:
* Hadoop Delegation Tokens and Block Tokens supported for Ozone.
* Transparent Data Encryption (TDE) Support - Allows data blocks to be encrypted-at-rest.
* Kerberos support for Ozone.
* Certificate Infrastructure for Ozone - Tokens use PKI instead of shared secrets.
* Datanode to Datanode communication secured via mutual TLS.
* Ability secure ozone cluster that works with Yarn, Hive, and Spark.
* Skaffold support to deploy Ozone clusters on K8s.
* Support S3 Authentication Mechanisms like - S3 v4 Authentication protocol.
* S3 Gateway supports Multipart upload.
* S3A file system is tested and supported.
* Support for Tracing and Profiling for all Ozone components.
* Audit Support - including Audit Parser tools.
* Apache Ranger Support in Ozone.
* Extensive failure testing for Ozone.
The RC artifacts are available at https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
Please try out<https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, vote, or just give us feedback.
The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
Thank you very much,
Ajay
Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Posted by Ajay Kumar <aj...@hortonworks.com>.
Hi Sandeep, Thanks for testing this. Will cut new RC with its fix included.
Best,
Ajay
From: Sandeep Nemuri <nh...@gmail.com>
Date: Wednesday, April 10, 2019 at 11:29 AM
To: Ajay Kumar <aj...@hortonworks.com>
Cc: Hadoop Common <co...@hadoop.apache.org>, Hdfs-dev <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Subject: Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Thanks Ajay for working on release.
We've found an issue in DN where the root filesystem is getting filled after running the cluster for a while. This is because after importing a container successfully, datanode is not deleting the container tar.gz file from working directory(/tmp by default).
More details in https://issues.apache.org/jira/browse/HDDS-1417
Thanks
Sandeep
On Tue, Apr 9, 2019 at 3:55 AM Ajay Kumar <aj...@hortonworks.com>> wrote:
Hi all,
We have created the first release candidate (RC0) for Apache Hadoop Ozone 0.4.0-alpha.
This release contains security payload for Ozone. Below are some important features in it:
* Hadoop Delegation Tokens and Block Tokens supported for Ozone.
* Transparent Data Encryption (TDE) Support - Allows data blocks to be encrypted-at-rest.
* Kerberos support for Ozone.
* Certificate Infrastructure for Ozone - Tokens use PKI instead of shared secrets.
* Datanode to Datanode communication secured via mutual TLS.
* Ability secure ozone cluster that works with Yarn, Hive, and Spark.
* Skaffold support to deploy Ozone clusters on K8s.
* Support S3 Authentication Mechanisms like - S3 v4 Authentication protocol.
* S3 Gateway supports Multipart upload.
* S3A file system is tested and supported.
* Support for Tracing and Profiling for all Ozone components.
* Audit Support - including Audit Parser tools.
* Apache Ranger Support in Ozone.
* Extensive failure testing for Ozone.
The RC artifacts are available at https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
Please try out<https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, vote, or just give us feedback.
The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
Thank you very much,
Ajay
--
Regards
Sandeep Nemuri
Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Posted by Ajay Kumar <aj...@hortonworks.com>.
Hi Sandeep, Thanks for testing this. Will cut new RC with its fix included.
Best,
Ajay
From: Sandeep Nemuri <nh...@gmail.com>
Date: Wednesday, April 10, 2019 at 11:29 AM
To: Ajay Kumar <aj...@hortonworks.com>
Cc: Hadoop Common <co...@hadoop.apache.org>, Hdfs-dev <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Subject: Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Thanks Ajay for working on release.
We've found an issue in DN where the root filesystem is getting filled after running the cluster for a while. This is because after importing a container successfully, datanode is not deleting the container tar.gz file from working directory(/tmp by default).
More details in https://issues.apache.org/jira/browse/HDDS-1417
Thanks
Sandeep
On Tue, Apr 9, 2019 at 3:55 AM Ajay Kumar <aj...@hortonworks.com>> wrote:
Hi all,
We have created the first release candidate (RC0) for Apache Hadoop Ozone 0.4.0-alpha.
This release contains security payload for Ozone. Below are some important features in it:
* Hadoop Delegation Tokens and Block Tokens supported for Ozone.
* Transparent Data Encryption (TDE) Support - Allows data blocks to be encrypted-at-rest.
* Kerberos support for Ozone.
* Certificate Infrastructure for Ozone - Tokens use PKI instead of shared secrets.
* Datanode to Datanode communication secured via mutual TLS.
* Ability secure ozone cluster that works with Yarn, Hive, and Spark.
* Skaffold support to deploy Ozone clusters on K8s.
* Support S3 Authentication Mechanisms like - S3 v4 Authentication protocol.
* S3 Gateway supports Multipart upload.
* S3A file system is tested and supported.
* Support for Tracing and Profiling for all Ozone components.
* Audit Support - including Audit Parser tools.
* Apache Ranger Support in Ozone.
* Extensive failure testing for Ozone.
The RC artifacts are available at https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
Please try out<https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, vote, or just give us feedback.
The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
Thank you very much,
Ajay
--
Regards
Sandeep Nemuri
Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Posted by Ajay Kumar <aj...@hortonworks.com>.
Hi Sandeep, Thanks for testing this. Will cut new RC with its fix included.
Best,
Ajay
From: Sandeep Nemuri <nh...@gmail.com>
Date: Wednesday, April 10, 2019 at 11:29 AM
To: Ajay Kumar <aj...@hortonworks.com>
Cc: Hadoop Common <co...@hadoop.apache.org>, Hdfs-dev <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Subject: Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Thanks Ajay for working on release.
We've found an issue in DN where the root filesystem is getting filled after running the cluster for a while. This is because after importing a container successfully, datanode is not deleting the container tar.gz file from working directory(/tmp by default).
More details in https://issues.apache.org/jira/browse/HDDS-1417
Thanks
Sandeep
On Tue, Apr 9, 2019 at 3:55 AM Ajay Kumar <aj...@hortonworks.com>> wrote:
Hi all,
We have created the first release candidate (RC0) for Apache Hadoop Ozone 0.4.0-alpha.
This release contains security payload for Ozone. Below are some important features in it:
* Hadoop Delegation Tokens and Block Tokens supported for Ozone.
* Transparent Data Encryption (TDE) Support - Allows data blocks to be encrypted-at-rest.
* Kerberos support for Ozone.
* Certificate Infrastructure for Ozone - Tokens use PKI instead of shared secrets.
* Datanode to Datanode communication secured via mutual TLS.
* Ability secure ozone cluster that works with Yarn, Hive, and Spark.
* Skaffold support to deploy Ozone clusters on K8s.
* Support S3 Authentication Mechanisms like - S3 v4 Authentication protocol.
* S3 Gateway supports Multipart upload.
* S3A file system is tested and supported.
* Support for Tracing and Profiling for all Ozone components.
* Audit Support - including Audit Parser tools.
* Apache Ranger Support in Ozone.
* Extensive failure testing for Ozone.
The RC artifacts are available at https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
Please try out<https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, vote, or just give us feedback.
The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
Thank you very much,
Ajay
--
Regards
Sandeep Nemuri
Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Posted by Ajay Kumar <aj...@hortonworks.com>.
Hi Sandeep, Thanks for testing this. Will cut new RC with its fix included.
Best,
Ajay
From: Sandeep Nemuri <nh...@gmail.com>
Date: Wednesday, April 10, 2019 at 11:29 AM
To: Ajay Kumar <aj...@hortonworks.com>
Cc: Hadoop Common <co...@hadoop.apache.org>, Hdfs-dev <hd...@hadoop.apache.org>, "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>, "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>
Subject: Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Thanks Ajay for working on release.
We've found an issue in DN where the root filesystem is getting filled after running the cluster for a while. This is because after importing a container successfully, datanode is not deleting the container tar.gz file from working directory(/tmp by default).
More details in https://issues.apache.org/jira/browse/HDDS-1417
Thanks
Sandeep
On Tue, Apr 9, 2019 at 3:55 AM Ajay Kumar <aj...@hortonworks.com>> wrote:
Hi all,
We have created the first release candidate (RC0) for Apache Hadoop Ozone 0.4.0-alpha.
This release contains security payload for Ozone. Below are some important features in it:
* Hadoop Delegation Tokens and Block Tokens supported for Ozone.
* Transparent Data Encryption (TDE) Support - Allows data blocks to be encrypted-at-rest.
* Kerberos support for Ozone.
* Certificate Infrastructure for Ozone - Tokens use PKI instead of shared secrets.
* Datanode to Datanode communication secured via mutual TLS.
* Ability secure ozone cluster that works with Yarn, Hive, and Spark.
* Skaffold support to deploy Ozone clusters on K8s.
* Support S3 Authentication Mechanisms like - S3 v4 Authentication protocol.
* S3 Gateway supports Multipart upload.
* S3A file system is tested and supported.
* Support for Tracing and Profiling for all Ozone components.
* Audit Support - including Audit Parser tools.
* Apache Ranger Support in Ozone.
* Extensive failure testing for Ozone.
The RC artifacts are available at https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
Please try out<https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, vote, or just give us feedback.
The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
Thank you very much,
Ajay
--
Regards
Sandeep Nemuri
Re: [VOTE]: Hadoop Ozone 0.4.0-alpha RC0 release
Posted by Sandeep Nemuri <nh...@gmail.com>.
Thanks Ajay for working on release.
We've found an issue in DN where the root filesystem is getting filled
after running the cluster for a while. This is because after importing a
container successfully, datanode is not deleting the container tar.gz file
from working directory(/tmp by default).
More details in https://issues.apache.org/jira/browse/HDDS-1417
Thanks
Sandeep
On Tue, Apr 9, 2019 at 3:55 AM Ajay Kumar <aj...@hortonworks.com>
wrote:
> Hi all,
>
>
>
> We have created the first release candidate (RC0) for Apache Hadoop Ozone
> 0.4.0-alpha.
>
>
>
> This release contains security payload for Ozone. Below are some important
> features in it:
>
> * Hadoop Delegation Tokens and Block Tokens supported for Ozone.
> * Transparent Data Encryption (TDE) Support - Allows data blocks to be
> encrypted-at-rest.
> * Kerberos support for Ozone.
> * Certificate Infrastructure for Ozone - Tokens use PKI instead of
> shared secrets.
> * Datanode to Datanode communication secured via mutual TLS.
> * Ability secure ozone cluster that works with Yarn, Hive, and Spark.
> * Skaffold support to deploy Ozone clusters on K8s.
> * Support S3 Authentication Mechanisms like - S3 v4 Authentication
> protocol.
> * S3 Gateway supports Multipart upload.
> * S3A file system is tested and supported.
> * Support for Tracing and Profiling for all Ozone components.
> * Audit Support - including Audit Parser tools.
> * Apache Ranger Support in Ozone.
> * Extensive failure testing for Ozone.
>
>
> The RC artifacts are available at
> https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc0
>
>
>
> The RC tag in git is ozone-0.4.0-alpha-RC0 (git hash
> e5df6e94a5137b84cf7a6ee2e0ef7a8dae1b11a1)
>
>
>
> Please try out<
> https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>,
> vote, or just give us feedback.
>
>
>
> The vote will run for 5 days, ending on April 13, 2019, 19:00 UTC.
>
>
>
> Thank you very much,
>
> Ajay
>
>
--
* Regards*
* Sandeep Nemuri*