You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Steve Loughran <st...@cloudera.com.INVALID> on 2023/02/21 13:42:54 UTC
[VOTE] Release Apache Hadoop 3.3.5
Apache Hadoop 3.3.5
Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
What we would like is for anyone who can to verify the tarballs, especially
anyone who can try the arm64 binaries as we want to include them too.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
The git tag is release-3.3.5-RC1, commit 274f91a3259
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1368/
You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
Change log
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
Release notes
https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
This is off branch-3.3 and is the first big release since 3.3.2.
Key changes include
* Big update of dependencies to try and keep those reports of
transitive CVEs under control -both genuine and false positives.
* HDFS RBF enhancements
* Critical fix to ABFS input stream prefetching for correct reading.
* Vectored IO API for all FSDataInputStream implementations, with
high-performance versions for file:// and s3a:// filesystems.
file:// through java native io
s3a:// parallel GET requests.
* This release includes Arm64 binaries. Please can anyone with
compatible systems validate these.
Note, because the arm64 binaries are built separately on a different
platform and JVM, their jar files may not match those of the x86
release -and therefore the maven artifacts. I don't think this is
an issue (the ASF actually releases source tarballs, the binaries are
there for help only, though with the maven repo that's a bit blurred).
The only way to be consistent would actually untar the x86.tar.gz,
overwrite its binaries with the arm stuff, retar, sign and push out
for the vote. Even automating that would be risky.
Please try the release and vote. The vote will run for 5 days.
Steve and Mukund
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Ayush Saxena <ay...@gmail.com>.
>
> And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
Not sure if that is gonna help, People might ignore that or check that in
overconfidence. No harm though..
BTW Ozone has some cool stuff to handle this, it was added here:
https://github.com/apache/ozone/pull/2199
It checks for each PR, if the changes bring any new transitive dependency
or not and if it does, it flags that and then licence and all can be
managed. Worth exploring....
-Ayush
On Sat, 25 Feb 2023 at 01:09, Steve Loughran <st...@cloudera.com.invalid>
wrote:
> need this pr in too, https://github.com/apache/hadoop/pull/5429
>
> 1. cuts back on some transitive dependencies from hadoop-aliyun
> 2. fixes LICENSE-bin to be correct
>
> #2 is the blocker...and it looks like 3.2.x will also need fixup as well as
> the later ones -hadoop binaries have shipped without that file being up to
> date, but at least all the transitive stuff is correctly licensed. And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
>
> if this goes in, I will do the rebuild on monday UK time
>
> On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> >
> > And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> > maven releases (3.9.0)
> >
> > on a new local build with maven updated on homebrew (which i needed for
> > spark). so a code change too. That issue doesn't surface on our
> > release dockers, but will hit other people. especially over time. Going
> to
> > revert HADOOP-18590. Publish SBOM artifacts (#5281)
> >
> >
> >
> > On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com>
> wrote:
> >
> >> ok, let me cancel, update those jiras and kick off again. that will save
> >> anyone else having to do their homework
> >>
> >> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> >> wrote:
> >>
> >>> I'm now -1 as I found the wrong information on the top page (index.md).
> >>>
> >>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in
> HDFS
> >>> Router Based Federation.
> >>>
> >>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5
> before,
> >>> though it is actually not in branch-3.3. I corrected the fix version
> and
> >>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> >>> Unfortunately, it won't be fixed soon. I should have let you know at
> that
> >>> time, sorry. Supporting Observer NameNode in RBF is a prominent
> feature.
> >>> So I think we have to delete the description from the top page not to
> >>> confuse Hadoop users.
> >>>
> >>> - Takanobu
> >>>
> >>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> >>>
> >>> > Thanks for driving the release, Steve and Mukund.
> >>> >
> >>> > I found that there were some jiras with wrong fix versions.
> >>> >
> >>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> >>> > - HDFS-16845
> >>> > - HADOOP-18345
> >>> >
> >>> > The fix versions didn't include 3.3.5, but actually, it is in
> 3.3.5-RC1
> >>> > (and it is not in release-3.3.4) :
> >>> > - HADOOP-17276
> >>> > - HDFS-13293
> >>> > - HDFS-15630
> >>> > - HDFS-16266
> >>> > - HADOOP-18003
> >>> > - HDFS-16310
> >>> > - HADOOP-18014
> >>> >
> >>> > I corrected all the wrong fix versions just now. I'm not sure we
> should
> >>> > revote it since it only affects the changelog.
> >>> >
> >>> > - Takanobu
> >>> >
> >>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >>> >
> >>> >> Apache Hadoop 3.3.5
> >>> >>
> >>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> >>> 3.3.5.
> >>> >>
> >>> >> What we would like is for anyone who can to verify the tarballs,
> >>> >> especially
> >>> >> anyone who can try the arm64 binaries as we want to include them
> too.
> >>> >>
> >>> >> The RC is available at:
> >>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>> >>
> >>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>> >>
> >>> >> The maven artifacts are staged at
> >>> >>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>> >>
> >>> >> You can find my public key at:
> >>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>> >>
> >>> >> Change log
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>> >>
> >>> >> Release notes
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>> >>
> >>> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>> >>
> >>> >> Key changes include
> >>> >>
> >>> >> * Big update of dependencies to try and keep those reports of
> >>> >> transitive CVEs under control -both genuine and false positives.
> >>> >> * HDFS RBF enhancements
> >>> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >>> >> * Vectored IO API for all FSDataInputStream implementations, with
> >>> >> high-performance versions for file:// and s3a:// filesystems.
> >>> >> file:// through java native io
> >>> >> s3a:// parallel GET requests.
> >>> >> * This release includes Arm64 binaries. Please can anyone with
> >>> >> compatible systems validate these.
> >>> >>
> >>> >> Note, because the arm64 binaries are built separately on a different
> >>> >> platform and JVM, their jar files may not match those of the x86
> >>> >> release -and therefore the maven artifacts. I don't think this is
> >>> >> an issue (the ASF actually releases source tarballs, the binaries
> are
> >>> >> there for help only, though with the maven repo that's a bit
> blurred).
> >>> >>
> >>> >> The only way to be consistent would actually untar the x86.tar.gz,
> >>> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >>> >> for the vote. Even automating that would be risky.
> >>> >>
> >>> >> Please try the release and vote. The vote will run for 5 days.
> >>> >>
> >>> >> Steve and Mukund
> >>> >>
> >>> >
> >>>
> >>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Ayush Saxena <ay...@gmail.com>.
>
> And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
Not sure if that is gonna help, People might ignore that or check that in
overconfidence. No harm though..
BTW Ozone has some cool stuff to handle this, it was added here:
https://github.com/apache/ozone/pull/2199
It checks for each PR, if the changes bring any new transitive dependency
or not and if it does, it flags that and then licence and all can be
managed. Worth exploring....
-Ayush
On Sat, 25 Feb 2023 at 01:09, Steve Loughran <st...@cloudera.com.invalid>
wrote:
> need this pr in too, https://github.com/apache/hadoop/pull/5429
>
> 1. cuts back on some transitive dependencies from hadoop-aliyun
> 2. fixes LICENSE-bin to be correct
>
> #2 is the blocker...and it looks like 3.2.x will also need fixup as well as
> the later ones -hadoop binaries have shipped without that file being up to
> date, but at least all the transitive stuff is correctly licensed. And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
>
> if this goes in, I will do the rebuild on monday UK time
>
> On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> >
> > And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> > maven releases (3.9.0)
> >
> > on a new local build with maven updated on homebrew (which i needed for
> > spark). so a code change too. That issue doesn't surface on our
> > release dockers, but will hit other people. especially over time. Going
> to
> > revert HADOOP-18590. Publish SBOM artifacts (#5281)
> >
> >
> >
> > On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com>
> wrote:
> >
> >> ok, let me cancel, update those jiras and kick off again. that will save
> >> anyone else having to do their homework
> >>
> >> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> >> wrote:
> >>
> >>> I'm now -1 as I found the wrong information on the top page (index.md).
> >>>
> >>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in
> HDFS
> >>> Router Based Federation.
> >>>
> >>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5
> before,
> >>> though it is actually not in branch-3.3. I corrected the fix version
> and
> >>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> >>> Unfortunately, it won't be fixed soon. I should have let you know at
> that
> >>> time, sorry. Supporting Observer NameNode in RBF is a prominent
> feature.
> >>> So I think we have to delete the description from the top page not to
> >>> confuse Hadoop users.
> >>>
> >>> - Takanobu
> >>>
> >>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> >>>
> >>> > Thanks for driving the release, Steve and Mukund.
> >>> >
> >>> > I found that there were some jiras with wrong fix versions.
> >>> >
> >>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> >>> > - HDFS-16845
> >>> > - HADOOP-18345
> >>> >
> >>> > The fix versions didn't include 3.3.5, but actually, it is in
> 3.3.5-RC1
> >>> > (and it is not in release-3.3.4) :
> >>> > - HADOOP-17276
> >>> > - HDFS-13293
> >>> > - HDFS-15630
> >>> > - HDFS-16266
> >>> > - HADOOP-18003
> >>> > - HDFS-16310
> >>> > - HADOOP-18014
> >>> >
> >>> > I corrected all the wrong fix versions just now. I'm not sure we
> should
> >>> > revote it since it only affects the changelog.
> >>> >
> >>> > - Takanobu
> >>> >
> >>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >>> >
> >>> >> Apache Hadoop 3.3.5
> >>> >>
> >>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> >>> 3.3.5.
> >>> >>
> >>> >> What we would like is for anyone who can to verify the tarballs,
> >>> >> especially
> >>> >> anyone who can try the arm64 binaries as we want to include them
> too.
> >>> >>
> >>> >> The RC is available at:
> >>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>> >>
> >>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>> >>
> >>> >> The maven artifacts are staged at
> >>> >>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>> >>
> >>> >> You can find my public key at:
> >>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>> >>
> >>> >> Change log
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>> >>
> >>> >> Release notes
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>> >>
> >>> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>> >>
> >>> >> Key changes include
> >>> >>
> >>> >> * Big update of dependencies to try and keep those reports of
> >>> >> transitive CVEs under control -both genuine and false positives.
> >>> >> * HDFS RBF enhancements
> >>> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >>> >> * Vectored IO API for all FSDataInputStream implementations, with
> >>> >> high-performance versions for file:// and s3a:// filesystems.
> >>> >> file:// through java native io
> >>> >> s3a:// parallel GET requests.
> >>> >> * This release includes Arm64 binaries. Please can anyone with
> >>> >> compatible systems validate these.
> >>> >>
> >>> >> Note, because the arm64 binaries are built separately on a different
> >>> >> platform and JVM, their jar files may not match those of the x86
> >>> >> release -and therefore the maven artifacts. I don't think this is
> >>> >> an issue (the ASF actually releases source tarballs, the binaries
> are
> >>> >> there for help only, though with the maven repo that's a bit
> blurred).
> >>> >>
> >>> >> The only way to be consistent would actually untar the x86.tar.gz,
> >>> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >>> >> for the vote. Even automating that would be risky.
> >>> >>
> >>> >> Please try the release and vote. The vote will run for 5 days.
> >>> >>
> >>> >> Steve and Mukund
> >>> >>
> >>> >
> >>>
> >>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Ayush Saxena <ay...@gmail.com>.
>
> And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
Not sure if that is gonna help, People might ignore that or check that in
overconfidence. No harm though..
BTW Ozone has some cool stuff to handle this, it was added here:
https://github.com/apache/ozone/pull/2199
It checks for each PR, if the changes bring any new transitive dependency
or not and if it does, it flags that and then licence and all can be
managed. Worth exploring....
-Ayush
On Sat, 25 Feb 2023 at 01:09, Steve Loughran <st...@cloudera.com.invalid>
wrote:
> need this pr in too, https://github.com/apache/hadoop/pull/5429
>
> 1. cuts back on some transitive dependencies from hadoop-aliyun
> 2. fixes LICENSE-bin to be correct
>
> #2 is the blocker...and it looks like 3.2.x will also need fixup as well as
> the later ones -hadoop binaries have shipped without that file being up to
> date, but at least all the transitive stuff is correctly licensed. And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
>
> if this goes in, I will do the rebuild on monday UK time
>
> On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> >
> > And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> > maven releases (3.9.0)
> >
> > on a new local build with maven updated on homebrew (which i needed for
> > spark). so a code change too. That issue doesn't surface on our
> > release dockers, but will hit other people. especially over time. Going
> to
> > revert HADOOP-18590. Publish SBOM artifacts (#5281)
> >
> >
> >
> > On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com>
> wrote:
> >
> >> ok, let me cancel, update those jiras and kick off again. that will save
> >> anyone else having to do their homework
> >>
> >> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> >> wrote:
> >>
> >>> I'm now -1 as I found the wrong information on the top page (index.md).
> >>>
> >>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in
> HDFS
> >>> Router Based Federation.
> >>>
> >>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5
> before,
> >>> though it is actually not in branch-3.3. I corrected the fix version
> and
> >>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> >>> Unfortunately, it won't be fixed soon. I should have let you know at
> that
> >>> time, sorry. Supporting Observer NameNode in RBF is a prominent
> feature.
> >>> So I think we have to delete the description from the top page not to
> >>> confuse Hadoop users.
> >>>
> >>> - Takanobu
> >>>
> >>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> >>>
> >>> > Thanks for driving the release, Steve and Mukund.
> >>> >
> >>> > I found that there were some jiras with wrong fix versions.
> >>> >
> >>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> >>> > - HDFS-16845
> >>> > - HADOOP-18345
> >>> >
> >>> > The fix versions didn't include 3.3.5, but actually, it is in
> 3.3.5-RC1
> >>> > (and it is not in release-3.3.4) :
> >>> > - HADOOP-17276
> >>> > - HDFS-13293
> >>> > - HDFS-15630
> >>> > - HDFS-16266
> >>> > - HADOOP-18003
> >>> > - HDFS-16310
> >>> > - HADOOP-18014
> >>> >
> >>> > I corrected all the wrong fix versions just now. I'm not sure we
> should
> >>> > revote it since it only affects the changelog.
> >>> >
> >>> > - Takanobu
> >>> >
> >>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >>> >
> >>> >> Apache Hadoop 3.3.5
> >>> >>
> >>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> >>> 3.3.5.
> >>> >>
> >>> >> What we would like is for anyone who can to verify the tarballs,
> >>> >> especially
> >>> >> anyone who can try the arm64 binaries as we want to include them
> too.
> >>> >>
> >>> >> The RC is available at:
> >>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>> >>
> >>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>> >>
> >>> >> The maven artifacts are staged at
> >>> >>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>> >>
> >>> >> You can find my public key at:
> >>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>> >>
> >>> >> Change log
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>> >>
> >>> >> Release notes
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>> >>
> >>> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>> >>
> >>> >> Key changes include
> >>> >>
> >>> >> * Big update of dependencies to try and keep those reports of
> >>> >> transitive CVEs under control -both genuine and false positives.
> >>> >> * HDFS RBF enhancements
> >>> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >>> >> * Vectored IO API for all FSDataInputStream implementations, with
> >>> >> high-performance versions for file:// and s3a:// filesystems.
> >>> >> file:// through java native io
> >>> >> s3a:// parallel GET requests.
> >>> >> * This release includes Arm64 binaries. Please can anyone with
> >>> >> compatible systems validate these.
> >>> >>
> >>> >> Note, because the arm64 binaries are built separately on a different
> >>> >> platform and JVM, their jar files may not match those of the x86
> >>> >> release -and therefore the maven artifacts. I don't think this is
> >>> >> an issue (the ASF actually releases source tarballs, the binaries
> are
> >>> >> there for help only, though with the maven repo that's a bit
> blurred).
> >>> >>
> >>> >> The only way to be consistent would actually untar the x86.tar.gz,
> >>> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >>> >> for the vote. Even automating that would be risky.
> >>> >>
> >>> >> Please try the release and vote. The vote will run for 5 days.
> >>> >>
> >>> >> Steve and Mukund
> >>> >>
> >>> >
> >>>
> >>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Ayush Saxena <ay...@gmail.com>.
>
> And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
Not sure if that is gonna help, People might ignore that or check that in
overconfidence. No harm though..
BTW Ozone has some cool stuff to handle this, it was added here:
https://github.com/apache/ozone/pull/2199
It checks for each PR, if the changes bring any new transitive dependency
or not and if it does, it flags that and then licence and all can be
managed. Worth exploring....
-Ayush
On Sat, 25 Feb 2023 at 01:09, Steve Loughran <st...@cloudera.com.invalid>
wrote:
> need this pr in too, https://github.com/apache/hadoop/pull/5429
>
> 1. cuts back on some transitive dependencies from hadoop-aliyun
> 2. fixes LICENSE-bin to be correct
>
> #2 is the blocker...and it looks like 3.2.x will also need fixup as well as
> the later ones -hadoop binaries have shipped without that file being up to
> date, but at least all the transitive stuff is correctly licensed. And i
> think we need to change the PR template to mention transitive updates in
> the license bit too
>
> if this goes in, I will do the rebuild on monday UK time
>
> On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> >
> > And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> > maven releases (3.9.0)
> >
> > on a new local build with maven updated on homebrew (which i needed for
> > spark). so a code change too. That issue doesn't surface on our
> > release dockers, but will hit other people. especially over time. Going
> to
> > revert HADOOP-18590. Publish SBOM artifacts (#5281)
> >
> >
> >
> > On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com>
> wrote:
> >
> >> ok, let me cancel, update those jiras and kick off again. that will save
> >> anyone else having to do their homework
> >>
> >> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> >> wrote:
> >>
> >>> I'm now -1 as I found the wrong information on the top page (index.md).
> >>>
> >>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in
> HDFS
> >>> Router Based Federation.
> >>>
> >>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5
> before,
> >>> though it is actually not in branch-3.3. I corrected the fix version
> and
> >>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> >>> Unfortunately, it won't be fixed soon. I should have let you know at
> that
> >>> time, sorry. Supporting Observer NameNode in RBF is a prominent
> feature.
> >>> So I think we have to delete the description from the top page not to
> >>> confuse Hadoop users.
> >>>
> >>> - Takanobu
> >>>
> >>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> >>>
> >>> > Thanks for driving the release, Steve and Mukund.
> >>> >
> >>> > I found that there were some jiras with wrong fix versions.
> >>> >
> >>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> >>> > - HDFS-16845
> >>> > - HADOOP-18345
> >>> >
> >>> > The fix versions didn't include 3.3.5, but actually, it is in
> 3.3.5-RC1
> >>> > (and it is not in release-3.3.4) :
> >>> > - HADOOP-17276
> >>> > - HDFS-13293
> >>> > - HDFS-15630
> >>> > - HDFS-16266
> >>> > - HADOOP-18003
> >>> > - HDFS-16310
> >>> > - HADOOP-18014
> >>> >
> >>> > I corrected all the wrong fix versions just now. I'm not sure we
> should
> >>> > revote it since it only affects the changelog.
> >>> >
> >>> > - Takanobu
> >>> >
> >>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >>> >
> >>> >> Apache Hadoop 3.3.5
> >>> >>
> >>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> >>> 3.3.5.
> >>> >>
> >>> >> What we would like is for anyone who can to verify the tarballs,
> >>> >> especially
> >>> >> anyone who can try the arm64 binaries as we want to include them
> too.
> >>> >>
> >>> >> The RC is available at:
> >>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>> >>
> >>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>> >>
> >>> >> The maven artifacts are staged at
> >>> >>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>> >>
> >>> >> You can find my public key at:
> >>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>> >>
> >>> >> Change log
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>> >>
> >>> >> Release notes
> >>> >>
> >>> >>
> >>>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>> >>
> >>> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>> >>
> >>> >> Key changes include
> >>> >>
> >>> >> * Big update of dependencies to try and keep those reports of
> >>> >> transitive CVEs under control -both genuine and false positives.
> >>> >> * HDFS RBF enhancements
> >>> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >>> >> * Vectored IO API for all FSDataInputStream implementations, with
> >>> >> high-performance versions for file:// and s3a:// filesystems.
> >>> >> file:// through java native io
> >>> >> s3a:// parallel GET requests.
> >>> >> * This release includes Arm64 binaries. Please can anyone with
> >>> >> compatible systems validate these.
> >>> >>
> >>> >> Note, because the arm64 binaries are built separately on a different
> >>> >> platform and JVM, their jar files may not match those of the x86
> >>> >> release -and therefore the maven artifacts. I don't think this is
> >>> >> an issue (the ASF actually releases source tarballs, the binaries
> are
> >>> >> there for help only, though with the maven repo that's a bit
> blurred).
> >>> >>
> >>> >> The only way to be consistent would actually untar the x86.tar.gz,
> >>> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >>> >> for the vote. Even automating that would be risky.
> >>> >>
> >>> >> Please try the release and vote. The vote will run for 5 days.
> >>> >>
> >>> >> Steve and Mukund
> >>> >>
> >>> >
> >>>
> >>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
need this pr in too, https://github.com/apache/hadoop/pull/5429
1. cuts back on some transitive dependencies from hadoop-aliyun
2. fixes LICENSE-bin to be correct
#2 is the blocker...and it looks like 3.2.x will also need fixup as well as
the later ones -hadoop binaries have shipped without that file being up to
date, but at least all the transitive stuff is correctly licensed. And i
think we need to change the PR template to mention transitive updates in
the license bit too
if this goes in, I will do the rebuild on monday UK time
On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> maven releases (3.9.0)
>
> on a new local build with maven updated on homebrew (which i needed for
> spark). so a code change too. That issue doesn't surface on our
> release dockers, but will hit other people. especially over time. Going to
> revert HADOOP-18590. Publish SBOM artifacts (#5281)
>
>
>
> On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
>
>> ok, let me cancel, update those jiras and kick off again. that will save
>> anyone else having to do their homework
>>
>> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
>> wrote:
>>
>>> I'm now -1 as I found the wrong information on the top page (index.md).
>>>
>>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>>> Router Based Federation.
>>>
>>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>>> though it is actually not in branch-3.3. I corrected the fix version and
>>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>>> Unfortunately, it won't be fixed soon. I should have let you know at that
>>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>>> So I think we have to delete the description from the top page not to
>>> confuse Hadoop users.
>>>
>>> - Takanobu
>>>
>>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>>
>>> > Thanks for driving the release, Steve and Mukund.
>>> >
>>> > I found that there were some jiras with wrong fix versions.
>>> >
>>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>>> > - HDFS-16845
>>> > - HADOOP-18345
>>> >
>>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>>> > (and it is not in release-3.3.4) :
>>> > - HADOOP-17276
>>> > - HDFS-13293
>>> > - HDFS-15630
>>> > - HDFS-16266
>>> > - HADOOP-18003
>>> > - HDFS-16310
>>> > - HADOOP-18014
>>> >
>>> > I corrected all the wrong fix versions just now. I'm not sure we should
>>> > revote it since it only affects the changelog.
>>> >
>>> > - Takanobu
>>> >
>>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>>> >
>>> >> Apache Hadoop 3.3.5
>>> >>
>>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>>> 3.3.5.
>>> >>
>>> >> What we would like is for anyone who can to verify the tarballs,
>>> >> especially
>>> >> anyone who can try the arm64 binaries as we want to include them too.
>>> >>
>>> >> The RC is available at:
>>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>> >>
>>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>> >>
>>> >> The maven artifacts are staged at
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>> >>
>>> >> You can find my public key at:
>>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >>
>>> >> Change log
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>> >>
>>> >> Release notes
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>> >>
>>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>>> >>
>>> >> Key changes include
>>> >>
>>> >> * Big update of dependencies to try and keep those reports of
>>> >> transitive CVEs under control -both genuine and false positives.
>>> >> * HDFS RBF enhancements
>>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>>> >> * Vectored IO API for all FSDataInputStream implementations, with
>>> >> high-performance versions for file:// and s3a:// filesystems.
>>> >> file:// through java native io
>>> >> s3a:// parallel GET requests.
>>> >> * This release includes Arm64 binaries. Please can anyone with
>>> >> compatible systems validate these.
>>> >>
>>> >> Note, because the arm64 binaries are built separately on a different
>>> >> platform and JVM, their jar files may not match those of the x86
>>> >> release -and therefore the maven artifacts. I don't think this is
>>> >> an issue (the ASF actually releases source tarballs, the binaries are
>>> >> there for help only, though with the maven repo that's a bit blurred).
>>> >>
>>> >> The only way to be consistent would actually untar the x86.tar.gz,
>>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>>> >> for the vote. Even automating that would be risky.
>>> >>
>>> >> Please try the release and vote. The vote will run for 5 days.
>>> >>
>>> >> Steve and Mukund
>>> >>
>>> >
>>>
>>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
need this pr in too, https://github.com/apache/hadoop/pull/5429
1. cuts back on some transitive dependencies from hadoop-aliyun
2. fixes LICENSE-bin to be correct
#2 is the blocker...and it looks like 3.2.x will also need fixup as well as
the later ones -hadoop binaries have shipped without that file being up to
date, but at least all the transitive stuff is correctly licensed. And i
think we need to change the PR template to mention transitive updates in
the license bit too
if this goes in, I will do the rebuild on monday UK time
On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> maven releases (3.9.0)
>
> on a new local build with maven updated on homebrew (which i needed for
> spark). so a code change too. That issue doesn't surface on our
> release dockers, but will hit other people. especially over time. Going to
> revert HADOOP-18590. Publish SBOM artifacts (#5281)
>
>
>
> On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
>
>> ok, let me cancel, update those jiras and kick off again. that will save
>> anyone else having to do their homework
>>
>> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
>> wrote:
>>
>>> I'm now -1 as I found the wrong information on the top page (index.md).
>>>
>>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>>> Router Based Federation.
>>>
>>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>>> though it is actually not in branch-3.3. I corrected the fix version and
>>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>>> Unfortunately, it won't be fixed soon. I should have let you know at that
>>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>>> So I think we have to delete the description from the top page not to
>>> confuse Hadoop users.
>>>
>>> - Takanobu
>>>
>>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>>
>>> > Thanks for driving the release, Steve and Mukund.
>>> >
>>> > I found that there were some jiras with wrong fix versions.
>>> >
>>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>>> > - HDFS-16845
>>> > - HADOOP-18345
>>> >
>>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>>> > (and it is not in release-3.3.4) :
>>> > - HADOOP-17276
>>> > - HDFS-13293
>>> > - HDFS-15630
>>> > - HDFS-16266
>>> > - HADOOP-18003
>>> > - HDFS-16310
>>> > - HADOOP-18014
>>> >
>>> > I corrected all the wrong fix versions just now. I'm not sure we should
>>> > revote it since it only affects the changelog.
>>> >
>>> > - Takanobu
>>> >
>>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>>> >
>>> >> Apache Hadoop 3.3.5
>>> >>
>>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>>> 3.3.5.
>>> >>
>>> >> What we would like is for anyone who can to verify the tarballs,
>>> >> especially
>>> >> anyone who can try the arm64 binaries as we want to include them too.
>>> >>
>>> >> The RC is available at:
>>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>> >>
>>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>> >>
>>> >> The maven artifacts are staged at
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>> >>
>>> >> You can find my public key at:
>>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >>
>>> >> Change log
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>> >>
>>> >> Release notes
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>> >>
>>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>>> >>
>>> >> Key changes include
>>> >>
>>> >> * Big update of dependencies to try and keep those reports of
>>> >> transitive CVEs under control -both genuine and false positives.
>>> >> * HDFS RBF enhancements
>>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>>> >> * Vectored IO API for all FSDataInputStream implementations, with
>>> >> high-performance versions for file:// and s3a:// filesystems.
>>> >> file:// through java native io
>>> >> s3a:// parallel GET requests.
>>> >> * This release includes Arm64 binaries. Please can anyone with
>>> >> compatible systems validate these.
>>> >>
>>> >> Note, because the arm64 binaries are built separately on a different
>>> >> platform and JVM, their jar files may not match those of the x86
>>> >> release -and therefore the maven artifacts. I don't think this is
>>> >> an issue (the ASF actually releases source tarballs, the binaries are
>>> >> there for help only, though with the maven repo that's a bit blurred).
>>> >>
>>> >> The only way to be consistent would actually untar the x86.tar.gz,
>>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>>> >> for the vote. Even automating that would be risky.
>>> >>
>>> >> Please try the release and vote. The vote will run for 5 days.
>>> >>
>>> >> Steve and Mukund
>>> >>
>>> >
>>>
>>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
need this pr in too, https://github.com/apache/hadoop/pull/5429
1. cuts back on some transitive dependencies from hadoop-aliyun
2. fixes LICENSE-bin to be correct
#2 is the blocker...and it looks like 3.2.x will also need fixup as well as
the later ones -hadoop binaries have shipped without that file being up to
date, but at least all the transitive stuff is correctly licensed. And i
think we need to change the PR template to mention transitive updates in
the license bit too
if this goes in, I will do the rebuild on monday UK time
On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> maven releases (3.9.0)
>
> on a new local build with maven updated on homebrew (which i needed for
> spark). so a code change too. That issue doesn't surface on our
> release dockers, but will hit other people. especially over time. Going to
> revert HADOOP-18590. Publish SBOM artifacts (#5281)
>
>
>
> On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
>
>> ok, let me cancel, update those jiras and kick off again. that will save
>> anyone else having to do their homework
>>
>> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
>> wrote:
>>
>>> I'm now -1 as I found the wrong information on the top page (index.md).
>>>
>>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>>> Router Based Federation.
>>>
>>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>>> though it is actually not in branch-3.3. I corrected the fix version and
>>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>>> Unfortunately, it won't be fixed soon. I should have let you know at that
>>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>>> So I think we have to delete the description from the top page not to
>>> confuse Hadoop users.
>>>
>>> - Takanobu
>>>
>>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>>
>>> > Thanks for driving the release, Steve and Mukund.
>>> >
>>> > I found that there were some jiras with wrong fix versions.
>>> >
>>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>>> > - HDFS-16845
>>> > - HADOOP-18345
>>> >
>>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>>> > (and it is not in release-3.3.4) :
>>> > - HADOOP-17276
>>> > - HDFS-13293
>>> > - HDFS-15630
>>> > - HDFS-16266
>>> > - HADOOP-18003
>>> > - HDFS-16310
>>> > - HADOOP-18014
>>> >
>>> > I corrected all the wrong fix versions just now. I'm not sure we should
>>> > revote it since it only affects the changelog.
>>> >
>>> > - Takanobu
>>> >
>>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>>> >
>>> >> Apache Hadoop 3.3.5
>>> >>
>>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>>> 3.3.5.
>>> >>
>>> >> What we would like is for anyone who can to verify the tarballs,
>>> >> especially
>>> >> anyone who can try the arm64 binaries as we want to include them too.
>>> >>
>>> >> The RC is available at:
>>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>> >>
>>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>> >>
>>> >> The maven artifacts are staged at
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>> >>
>>> >> You can find my public key at:
>>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >>
>>> >> Change log
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>> >>
>>> >> Release notes
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>> >>
>>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>>> >>
>>> >> Key changes include
>>> >>
>>> >> * Big update of dependencies to try and keep those reports of
>>> >> transitive CVEs under control -both genuine and false positives.
>>> >> * HDFS RBF enhancements
>>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>>> >> * Vectored IO API for all FSDataInputStream implementations, with
>>> >> high-performance versions for file:// and s3a:// filesystems.
>>> >> file:// through java native io
>>> >> s3a:// parallel GET requests.
>>> >> * This release includes Arm64 binaries. Please can anyone with
>>> >> compatible systems validate these.
>>> >>
>>> >> Note, because the arm64 binaries are built separately on a different
>>> >> platform and JVM, their jar files may not match those of the x86
>>> >> release -and therefore the maven artifacts. I don't think this is
>>> >> an issue (the ASF actually releases source tarballs, the binaries are
>>> >> there for help only, though with the maven repo that's a bit blurred).
>>> >>
>>> >> The only way to be consistent would actually untar the x86.tar.gz,
>>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>>> >> for the vote. Even automating that would be risky.
>>> >>
>>> >> Please try the release and vote. The vote will run for 5 days.
>>> >>
>>> >> Steve and Mukund
>>> >>
>>> >
>>>
>>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
need this pr in too, https://github.com/apache/hadoop/pull/5429
1. cuts back on some transitive dependencies from hadoop-aliyun
2. fixes LICENSE-bin to be correct
#2 is the blocker...and it looks like 3.2.x will also need fixup as well as
the later ones -hadoop binaries have shipped without that file being up to
date, but at least all the transitive stuff is correctly licensed. And i
think we need to change the PR template to mention transitive updates in
the license bit too
if this goes in, I will do the rebuild on monday UK time
On Thu, 23 Feb 2023 at 11:18, Steve Loughran <st...@cloudera.com> wrote:
>
> And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
> maven releases (3.9.0)
>
> on a new local build with maven updated on homebrew (which i needed for
> spark). so a code change too. That issue doesn't surface on our
> release dockers, but will hit other people. especially over time. Going to
> revert HADOOP-18590. Publish SBOM artifacts (#5281)
>
>
>
> On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
>
>> ok, let me cancel, update those jiras and kick off again. that will save
>> anyone else having to do their homework
>>
>> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
>> wrote:
>>
>>> I'm now -1 as I found the wrong information on the top page (index.md).
>>>
>>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>>> Router Based Federation.
>>>
>>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>>> though it is actually not in branch-3.3. I corrected the fix version and
>>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>>> Unfortunately, it won't be fixed soon. I should have let you know at that
>>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>>> So I think we have to delete the description from the top page not to
>>> confuse Hadoop users.
>>>
>>> - Takanobu
>>>
>>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>>
>>> > Thanks for driving the release, Steve and Mukund.
>>> >
>>> > I found that there were some jiras with wrong fix versions.
>>> >
>>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>>> > - HDFS-16845
>>> > - HADOOP-18345
>>> >
>>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>>> > (and it is not in release-3.3.4) :
>>> > - HADOOP-17276
>>> > - HDFS-13293
>>> > - HDFS-15630
>>> > - HDFS-16266
>>> > - HADOOP-18003
>>> > - HDFS-16310
>>> > - HADOOP-18014
>>> >
>>> > I corrected all the wrong fix versions just now. I'm not sure we should
>>> > revote it since it only affects the changelog.
>>> >
>>> > - Takanobu
>>> >
>>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>>> >
>>> >> Apache Hadoop 3.3.5
>>> >>
>>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>>> 3.3.5.
>>> >>
>>> >> What we would like is for anyone who can to verify the tarballs,
>>> >> especially
>>> >> anyone who can try the arm64 binaries as we want to include them too.
>>> >>
>>> >> The RC is available at:
>>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>> >>
>>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>> >>
>>> >> The maven artifacts are staged at
>>> >>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>> >>
>>> >> You can find my public key at:
>>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>> >>
>>> >> Change log
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>> >>
>>> >> Release notes
>>> >>
>>> >>
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>> >>
>>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>>> >>
>>> >> Key changes include
>>> >>
>>> >> * Big update of dependencies to try and keep those reports of
>>> >> transitive CVEs under control -both genuine and false positives.
>>> >> * HDFS RBF enhancements
>>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>>> >> * Vectored IO API for all FSDataInputStream implementations, with
>>> >> high-performance versions for file:// and s3a:// filesystems.
>>> >> file:// through java native io
>>> >> s3a:// parallel GET requests.
>>> >> * This release includes Arm64 binaries. Please can anyone with
>>> >> compatible systems validate these.
>>> >>
>>> >> Note, because the arm64 binaries are built separately on a different
>>> >> platform and JVM, their jar files may not match those of the x86
>>> >> release -and therefore the maven artifacts. I don't think this is
>>> >> an issue (the ASF actually releases source tarballs, the binaries are
>>> >> there for help only, though with the maven repo that's a bit blurred).
>>> >>
>>> >> The only way to be consistent would actually untar the x86.tar.gz,
>>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>>> >> for the vote. Even automating that would be risky.
>>> >>
>>> >> Please try the release and vote. The vote will run for 5 days.
>>> >>
>>> >> Steve and Mukund
>>> >>
>>> >
>>>
>>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
maven releases (3.9.0)
on a new local build with maven updated on homebrew (which i needed for
spark). so a code change too. That issue doesn't surface on our
release dockers, but will hit other people. especially over time. Going to
revert HADOOP-18590. Publish SBOM artifacts (#5281)
On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
> ok, let me cancel, update those jiras and kick off again. that will save
> anyone else having to do their homework
>
> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> wrote:
>
>> I'm now -1 as I found the wrong information on the top page (index.md).
>>
>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>> Router Based Federation.
>>
>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>> though it is actually not in branch-3.3. I corrected the fix version and
>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>> Unfortunately, it won't be fixed soon. I should have let you know at that
>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>> So I think we have to delete the description from the top page not to
>> confuse Hadoop users.
>>
>> - Takanobu
>>
>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>
>> > Thanks for driving the release, Steve and Mukund.
>> >
>> > I found that there were some jiras with wrong fix versions.
>> >
>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>> > - HDFS-16845
>> > - HADOOP-18345
>> >
>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>> > (and it is not in release-3.3.4) :
>> > - HADOOP-17276
>> > - HDFS-13293
>> > - HDFS-15630
>> > - HDFS-16266
>> > - HADOOP-18003
>> > - HDFS-16310
>> > - HADOOP-18014
>> >
>> > I corrected all the wrong fix versions just now. I'm not sure we should
>> > revote it since it only affects the changelog.
>> >
>> > - Takanobu
>> >
>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>> >
>> >> Apache Hadoop 3.3.5
>> >>
>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>> 3.3.5.
>> >>
>> >> What we would like is for anyone who can to verify the tarballs,
>> >> especially
>> >> anyone who can try the arm64 binaries as we want to include them too.
>> >>
>> >> The RC is available at:
>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>> >>
>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>> >>
>> >> The maven artifacts are staged at
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>> >>
>> >> You can find my public key at:
>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >>
>> >> Change log
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>> >>
>> >> Release notes
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>> >>
>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>> >>
>> >> Key changes include
>> >>
>> >> * Big update of dependencies to try and keep those reports of
>> >> transitive CVEs under control -both genuine and false positives.
>> >> * HDFS RBF enhancements
>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>> >> * Vectored IO API for all FSDataInputStream implementations, with
>> >> high-performance versions for file:// and s3a:// filesystems.
>> >> file:// through java native io
>> >> s3a:// parallel GET requests.
>> >> * This release includes Arm64 binaries. Please can anyone with
>> >> compatible systems validate these.
>> >>
>> >> Note, because the arm64 binaries are built separately on a different
>> >> platform and JVM, their jar files may not match those of the x86
>> >> release -and therefore the maven artifacts. I don't think this is
>> >> an issue (the ASF actually releases source tarballs, the binaries are
>> >> there for help only, though with the maven repo that's a bit blurred).
>> >>
>> >> The only way to be consistent would actually untar the x86.tar.gz,
>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>> >> for the vote. Even automating that would be risky.
>> >>
>> >> Please try the release and vote. The vote will run for 5 days.
>> >>
>> >> Steve and Mukund
>> >>
>> >
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
maven releases (3.9.0)
on a new local build with maven updated on homebrew (which i needed for
spark). so a code change too. That issue doesn't surface on our
release dockers, but will hit other people. especially over time. Going to
revert HADOOP-18590. Publish SBOM artifacts (#5281)
On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
> ok, let me cancel, update those jiras and kick off again. that will save
> anyone else having to do their homework
>
> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> wrote:
>
>> I'm now -1 as I found the wrong information on the top page (index.md).
>>
>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>> Router Based Federation.
>>
>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>> though it is actually not in branch-3.3. I corrected the fix version and
>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>> Unfortunately, it won't be fixed soon. I should have let you know at that
>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>> So I think we have to delete the description from the top page not to
>> confuse Hadoop users.
>>
>> - Takanobu
>>
>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>
>> > Thanks for driving the release, Steve and Mukund.
>> >
>> > I found that there were some jiras with wrong fix versions.
>> >
>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>> > - HDFS-16845
>> > - HADOOP-18345
>> >
>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>> > (and it is not in release-3.3.4) :
>> > - HADOOP-17276
>> > - HDFS-13293
>> > - HDFS-15630
>> > - HDFS-16266
>> > - HADOOP-18003
>> > - HDFS-16310
>> > - HADOOP-18014
>> >
>> > I corrected all the wrong fix versions just now. I'm not sure we should
>> > revote it since it only affects the changelog.
>> >
>> > - Takanobu
>> >
>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>> >
>> >> Apache Hadoop 3.3.5
>> >>
>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>> 3.3.5.
>> >>
>> >> What we would like is for anyone who can to verify the tarballs,
>> >> especially
>> >> anyone who can try the arm64 binaries as we want to include them too.
>> >>
>> >> The RC is available at:
>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>> >>
>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>> >>
>> >> The maven artifacts are staged at
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>> >>
>> >> You can find my public key at:
>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >>
>> >> Change log
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>> >>
>> >> Release notes
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>> >>
>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>> >>
>> >> Key changes include
>> >>
>> >> * Big update of dependencies to try and keep those reports of
>> >> transitive CVEs under control -both genuine and false positives.
>> >> * HDFS RBF enhancements
>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>> >> * Vectored IO API for all FSDataInputStream implementations, with
>> >> high-performance versions for file:// and s3a:// filesystems.
>> >> file:// through java native io
>> >> s3a:// parallel GET requests.
>> >> * This release includes Arm64 binaries. Please can anyone with
>> >> compatible systems validate these.
>> >>
>> >> Note, because the arm64 binaries are built separately on a different
>> >> platform and JVM, their jar files may not match those of the x86
>> >> release -and therefore the maven artifacts. I don't think this is
>> >> an issue (the ASF actually releases source tarballs, the binaries are
>> >> there for help only, though with the maven repo that's a bit blurred).
>> >>
>> >> The only way to be consistent would actually untar the x86.tar.gz,
>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>> >> for the vote. Even automating that would be risky.
>> >>
>> >> Please try the release and vote. The vote will run for 5 days.
>> >>
>> >> Steve and Mukund
>> >>
>> >
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
maven releases (3.9.0)
on a new local build with maven updated on homebrew (which i needed for
spark). so a code change too. That issue doesn't surface on our
release dockers, but will hit other people. especially over time. Going to
revert HADOOP-18590. Publish SBOM artifacts (#5281)
On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
> ok, let me cancel, update those jiras and kick off again. that will save
> anyone else having to do their homework
>
> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> wrote:
>
>> I'm now -1 as I found the wrong information on the top page (index.md).
>>
>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>> Router Based Federation.
>>
>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>> though it is actually not in branch-3.3. I corrected the fix version and
>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>> Unfortunately, it won't be fixed soon. I should have let you know at that
>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>> So I think we have to delete the description from the top page not to
>> confuse Hadoop users.
>>
>> - Takanobu
>>
>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>
>> > Thanks for driving the release, Steve and Mukund.
>> >
>> > I found that there were some jiras with wrong fix versions.
>> >
>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>> > - HDFS-16845
>> > - HADOOP-18345
>> >
>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>> > (and it is not in release-3.3.4) :
>> > - HADOOP-17276
>> > - HDFS-13293
>> > - HDFS-15630
>> > - HDFS-16266
>> > - HADOOP-18003
>> > - HDFS-16310
>> > - HADOOP-18014
>> >
>> > I corrected all the wrong fix versions just now. I'm not sure we should
>> > revote it since it only affects the changelog.
>> >
>> > - Takanobu
>> >
>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>> >
>> >> Apache Hadoop 3.3.5
>> >>
>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>> 3.3.5.
>> >>
>> >> What we would like is for anyone who can to verify the tarballs,
>> >> especially
>> >> anyone who can try the arm64 binaries as we want to include them too.
>> >>
>> >> The RC is available at:
>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>> >>
>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>> >>
>> >> The maven artifacts are staged at
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>> >>
>> >> You can find my public key at:
>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >>
>> >> Change log
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>> >>
>> >> Release notes
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>> >>
>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>> >>
>> >> Key changes include
>> >>
>> >> * Big update of dependencies to try and keep those reports of
>> >> transitive CVEs under control -both genuine and false positives.
>> >> * HDFS RBF enhancements
>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>> >> * Vectored IO API for all FSDataInputStream implementations, with
>> >> high-performance versions for file:// and s3a:// filesystems.
>> >> file:// through java native io
>> >> s3a:// parallel GET requests.
>> >> * This release includes Arm64 binaries. Please can anyone with
>> >> compatible systems validate these.
>> >>
>> >> Note, because the arm64 binaries are built separately on a different
>> >> platform and JVM, their jar files may not match those of the x86
>> >> release -and therefore the maven artifacts. I don't think this is
>> >> an issue (the ASF actually releases source tarballs, the binaries are
>> >> there for help only, though with the maven repo that's a bit blurred).
>> >>
>> >> The only way to be consistent would actually untar the x86.tar.gz,
>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>> >> for the vote. Even automating that would be risky.
>> >>
>> >> Please try the release and vote. The vote will run for 5 days.
>> >>
>> >> Steve and Mukund
>> >>
>> >
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
maven releases (3.9.0)
on a new local build with maven updated on homebrew (which i needed for
spark). so a code change too. That issue doesn't surface on our
release dockers, but will hit other people. especially over time. Going to
revert HADOOP-18590. Publish SBOM artifacts (#5281)
On Thu, 23 Feb 2023 at 10:29, Steve Loughran <st...@cloudera.com> wrote:
> ok, let me cancel, update those jiras and kick off again. that will save
> anyone else having to do their homework
>
> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org>
> wrote:
>
>> I'm now -1 as I found the wrong information on the top page (index.md).
>>
>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>> Router Based Federation.
>>
>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>> though it is actually not in branch-3.3. I corrected the fix version and
>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>> Unfortunately, it won't be fixed soon. I should have let you know at that
>> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
>> So I think we have to delete the description from the top page not to
>> confuse Hadoop users.
>>
>> - Takanobu
>>
>> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>>
>> > Thanks for driving the release, Steve and Mukund.
>> >
>> > I found that there were some jiras with wrong fix versions.
>> >
>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>> > - HDFS-16845
>> > - HADOOP-18345
>> >
>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>> > (and it is not in release-3.3.4) :
>> > - HADOOP-17276
>> > - HDFS-13293
>> > - HDFS-15630
>> > - HDFS-16266
>> > - HADOOP-18003
>> > - HDFS-16310
>> > - HADOOP-18014
>> >
>> > I corrected all the wrong fix versions just now. I'm not sure we should
>> > revote it since it only affects the changelog.
>> >
>> > - Takanobu
>> >
>> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>> >
>> >> Apache Hadoop 3.3.5
>> >>
>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>> 3.3.5.
>> >>
>> >> What we would like is for anyone who can to verify the tarballs,
>> >> especially
>> >> anyone who can try the arm64 binaries as we want to include them too.
>> >>
>> >> The RC is available at:
>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>> >>
>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>> >>
>> >> The maven artifacts are staged at
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>> >>
>> >> You can find my public key at:
>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >>
>> >> Change log
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>> >>
>> >> Release notes
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>> >>
>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>> >>
>> >> Key changes include
>> >>
>> >> * Big update of dependencies to try and keep those reports of
>> >> transitive CVEs under control -both genuine and false positives.
>> >> * HDFS RBF enhancements
>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>> >> * Vectored IO API for all FSDataInputStream implementations, with
>> >> high-performance versions for file:// and s3a:// filesystems.
>> >> file:// through java native io
>> >> s3a:// parallel GET requests.
>> >> * This release includes Arm64 binaries. Please can anyone with
>> >> compatible systems validate these.
>> >>
>> >> Note, because the arm64 binaries are built separately on a different
>> >> platform and JVM, their jar files may not match those of the x86
>> >> release -and therefore the maven artifacts. I don't think this is
>> >> an issue (the ASF actually releases source tarballs, the binaries are
>> >> there for help only, though with the maven repo that's a bit blurred).
>> >>
>> >> The only way to be consistent would actually untar the x86.tar.gz,
>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>> >> for the vote. Even automating that would be risky.
>> >>
>> >> Please try the release and vote. The vote will run for 5 days.
>> >>
>> >> Steve and Mukund
>> >>
>> >
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
ok, let me cancel, update those jiras and kick off again. that will save
anyone else having to do their homework
On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org> wrote:
> I'm now -1 as I found the wrong information on the top page (index.md).
>
> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
> Router Based Federation.
>
> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
> though it is actually not in branch-3.3. I corrected the fix version and
> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> Unfortunately, it won't be fixed soon. I should have let you know at that
> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
> So I think we have to delete the description from the top page not to
> confuse Hadoop users.
>
> - Takanobu
>
> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>
> > Thanks for driving the release, Steve and Mukund.
> >
> > I found that there were some jiras with wrong fix versions.
> >
> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> > - HDFS-16845
> > - HADOOP-18345
> >
> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> > (and it is not in release-3.3.4) :
> > - HADOOP-17276
> > - HDFS-13293
> > - HDFS-15630
> > - HDFS-16266
> > - HADOOP-18003
> > - HDFS-16310
> > - HADOOP-18014
> >
> > I corrected all the wrong fix versions just now. I'm not sure we should
> > revote it since it only affects the changelog.
> >
> > - Takanobu
> >
> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >
> >> Apache Hadoop 3.3.5
> >>
> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> 3.3.5.
> >>
> >> What we would like is for anyone who can to verify the tarballs,
> >> especially
> >> anyone who can try the arm64 binaries as we want to include them too.
> >>
> >> The RC is available at:
> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>
> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>
> >> The maven artifacts are staged at
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>
> >> You can find my public key at:
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >> Change log
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>
> >> Release notes
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>
> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>
> >> Key changes include
> >>
> >> * Big update of dependencies to try and keep those reports of
> >> transitive CVEs under control -both genuine and false positives.
> >> * HDFS RBF enhancements
> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >> * Vectored IO API for all FSDataInputStream implementations, with
> >> high-performance versions for file:// and s3a:// filesystems.
> >> file:// through java native io
> >> s3a:// parallel GET requests.
> >> * This release includes Arm64 binaries. Please can anyone with
> >> compatible systems validate these.
> >>
> >> Note, because the arm64 binaries are built separately on a different
> >> platform and JVM, their jar files may not match those of the x86
> >> release -and therefore the maven artifacts. I don't think this is
> >> an issue (the ASF actually releases source tarballs, the binaries are
> >> there for help only, though with the maven repo that's a bit blurred).
> >>
> >> The only way to be consistent would actually untar the x86.tar.gz,
> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >> for the vote. Even automating that would be risky.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> Steve and Mukund
> >>
> >
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
ok, let me cancel, update those jiras and kick off again. that will save
anyone else having to do their homework
On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org> wrote:
> I'm now -1 as I found the wrong information on the top page (index.md).
>
> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
> Router Based Federation.
>
> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
> though it is actually not in branch-3.3. I corrected the fix version and
> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> Unfortunately, it won't be fixed soon. I should have let you know at that
> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
> So I think we have to delete the description from the top page not to
> confuse Hadoop users.
>
> - Takanobu
>
> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>
> > Thanks for driving the release, Steve and Mukund.
> >
> > I found that there were some jiras with wrong fix versions.
> >
> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> > - HDFS-16845
> > - HADOOP-18345
> >
> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> > (and it is not in release-3.3.4) :
> > - HADOOP-17276
> > - HDFS-13293
> > - HDFS-15630
> > - HDFS-16266
> > - HADOOP-18003
> > - HDFS-16310
> > - HADOOP-18014
> >
> > I corrected all the wrong fix versions just now. I'm not sure we should
> > revote it since it only affects the changelog.
> >
> > - Takanobu
> >
> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >
> >> Apache Hadoop 3.3.5
> >>
> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> 3.3.5.
> >>
> >> What we would like is for anyone who can to verify the tarballs,
> >> especially
> >> anyone who can try the arm64 binaries as we want to include them too.
> >>
> >> The RC is available at:
> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>
> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>
> >> The maven artifacts are staged at
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>
> >> You can find my public key at:
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >> Change log
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>
> >> Release notes
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>
> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>
> >> Key changes include
> >>
> >> * Big update of dependencies to try and keep those reports of
> >> transitive CVEs under control -both genuine and false positives.
> >> * HDFS RBF enhancements
> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >> * Vectored IO API for all FSDataInputStream implementations, with
> >> high-performance versions for file:// and s3a:// filesystems.
> >> file:// through java native io
> >> s3a:// parallel GET requests.
> >> * This release includes Arm64 binaries. Please can anyone with
> >> compatible systems validate these.
> >>
> >> Note, because the arm64 binaries are built separately on a different
> >> platform and JVM, their jar files may not match those of the x86
> >> release -and therefore the maven artifacts. I don't think this is
> >> an issue (the ASF actually releases source tarballs, the binaries are
> >> there for help only, though with the maven repo that's a bit blurred).
> >>
> >> The only way to be consistent would actually untar the x86.tar.gz,
> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >> for the vote. Even automating that would be risky.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> Steve and Mukund
> >>
> >
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
ok, let me cancel, update those jiras and kick off again. that will save
anyone else having to do their homework
On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org> wrote:
> I'm now -1 as I found the wrong information on the top page (index.md).
>
> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
> Router Based Federation.
>
> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
> though it is actually not in branch-3.3. I corrected the fix version and
> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> Unfortunately, it won't be fixed soon. I should have let you know at that
> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
> So I think we have to delete the description from the top page not to
> confuse Hadoop users.
>
> - Takanobu
>
> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>
> > Thanks for driving the release, Steve and Mukund.
> >
> > I found that there were some jiras with wrong fix versions.
> >
> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> > - HDFS-16845
> > - HADOOP-18345
> >
> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> > (and it is not in release-3.3.4) :
> > - HADOOP-17276
> > - HDFS-13293
> > - HDFS-15630
> > - HDFS-16266
> > - HADOOP-18003
> > - HDFS-16310
> > - HADOOP-18014
> >
> > I corrected all the wrong fix versions just now. I'm not sure we should
> > revote it since it only affects the changelog.
> >
> > - Takanobu
> >
> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >
> >> Apache Hadoop 3.3.5
> >>
> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> 3.3.5.
> >>
> >> What we would like is for anyone who can to verify the tarballs,
> >> especially
> >> anyone who can try the arm64 binaries as we want to include them too.
> >>
> >> The RC is available at:
> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>
> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>
> >> The maven artifacts are staged at
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>
> >> You can find my public key at:
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >> Change log
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>
> >> Release notes
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>
> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>
> >> Key changes include
> >>
> >> * Big update of dependencies to try and keep those reports of
> >> transitive CVEs under control -both genuine and false positives.
> >> * HDFS RBF enhancements
> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >> * Vectored IO API for all FSDataInputStream implementations, with
> >> high-performance versions for file:// and s3a:// filesystems.
> >> file:// through java native io
> >> s3a:// parallel GET requests.
> >> * This release includes Arm64 binaries. Please can anyone with
> >> compatible systems validate these.
> >>
> >> Note, because the arm64 binaries are built separately on a different
> >> platform and JVM, their jar files may not match those of the x86
> >> release -and therefore the maven artifacts. I don't think this is
> >> an issue (the ASF actually releases source tarballs, the binaries are
> >> there for help only, though with the maven repo that's a bit blurred).
> >>
> >> The only way to be consistent would actually untar the x86.tar.gz,
> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >> for the vote. Even automating that would be risky.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> Steve and Mukund
> >>
> >
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
ok, let me cancel, update those jiras and kick off again. that will save
anyone else having to do their homework
On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma <ta...@apache.org> wrote:
> I'm now -1 as I found the wrong information on the top page (index.md).
>
> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
> Router Based Federation.
>
> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
> though it is actually not in branch-3.3. I corrected the fix version and
> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> Unfortunately, it won't be fixed soon. I should have let you know at that
> time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
> So I think we have to delete the description from the top page not to
> confuse Hadoop users.
>
> - Takanobu
>
> 2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
>
> > Thanks for driving the release, Steve and Mukund.
> >
> > I found that there were some jiras with wrong fix versions.
> >
> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> > - HDFS-16845
> > - HADOOP-18345
> >
> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> > (and it is not in release-3.3.4) :
> > - HADOOP-17276
> > - HDFS-13293
> > - HDFS-15630
> > - HDFS-16266
> > - HADOOP-18003
> > - HDFS-16310
> > - HADOOP-18014
> >
> > I corrected all the wrong fix versions just now. I'm not sure we should
> > revote it since it only affects the changelog.
> >
> > - Takanobu
> >
> > 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> >
> >> Apache Hadoop 3.3.5
> >>
> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> 3.3.5.
> >>
> >> What we would like is for anyone who can to verify the tarballs,
> >> especially
> >> anyone who can try the arm64 binaries as we want to include them too.
> >>
> >> The RC is available at:
> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>
> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>
> >> The maven artifacts are staged at
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>
> >> You can find my public key at:
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >> Change log
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>
> >> Release notes
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>
> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>
> >> Key changes include
> >>
> >> * Big update of dependencies to try and keep those reports of
> >> transitive CVEs under control -both genuine and false positives.
> >> * HDFS RBF enhancements
> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >> * Vectored IO API for all FSDataInputStream implementations, with
> >> high-performance versions for file:// and s3a:// filesystems.
> >> file:// through java native io
> >> s3a:// parallel GET requests.
> >> * This release includes Arm64 binaries. Please can anyone with
> >> compatible systems validate these.
> >>
> >> Note, because the arm64 binaries are built separately on a different
> >> platform and JVM, their jar files may not match those of the x86
> >> release -and therefore the maven artifacts. I don't think this is
> >> an issue (the ASF actually releases source tarballs, the binaries are
> >> there for help only, though with the maven repo that's a bit blurred).
> >>
> >> The only way to be consistent would actually untar the x86.tar.gz,
> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >> for the vote. Even automating that would be risky.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> Steve and Mukund
> >>
> >
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
I'm now -1 as I found the wrong information on the top page (index.md).
> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.
The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.
- Takanobu
2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>> transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>> high-performance versions for file:// and s3a:// filesystems.
>> file:// through java native io
>> s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>> compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
I'm now -1 as I found the wrong information on the top page (index.md).
> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.
The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.
- Takanobu
2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>> transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>> high-performance versions for file:// and s3a:// filesystems.
>> file:// through java native io
>> s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>> compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
I'm now -1 as I found the wrong information on the top page (index.md).
> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.
The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.
- Takanobu
2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>> transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>> high-performance versions for file:// and s3a:// filesystems.
>> file:// through java native io
>> s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>> compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
I'm now -1 as I found the wrong information on the top page (index.md).
> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.
The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry. Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.
- Takanobu
2023年2月23日(木) 17:17 Takanobu Asanuma <ta...@apache.org>:
> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>> transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>> high-performance versions for file:// and s3a:// filesystems.
>> file:// through java native io
>> s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>> compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
Thanks for driving the release, Steve and Mukund.
I found that there were some jiras with wrong fix versions.
The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345
The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014
I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.
- Takanobu
2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
> transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
> high-performance versions for file:// and s3a:// filesystems.
> file:// through java native io
> s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
> compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
Thanks for driving the release, Steve and Mukund.
I found that there were some jiras with wrong fix versions.
The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345
The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014
I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.
- Takanobu
2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
> transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
> high-performance versions for file:// and s3a:// filesystems.
> file:// through java native io
> s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
> compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
Thanks for driving the release, Steve and Mukund.
I found that there were some jiras with wrong fix versions.
The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345
The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014
I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.
- Takanobu
2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
> transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
> high-performance versions for file:// and s3a:// filesystems.
> file:// through java native io
> s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
> compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>
Re: [VOTE] Release Apache Hadoop 3.3.5
Posted by Takanobu Asanuma <ta...@apache.org>.
Thanks for driving the release, Steve and Mukund.
I found that there were some jiras with wrong fix versions.
The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345
The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014
I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.
- Takanobu
2023年2月21日(火) 22:43 Steve Loughran <st...@cloudera.com.invalid>:
> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
> transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
> high-performance versions for file:// and s3a:// filesystems.
> file:// through java native io
> s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
> compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>