You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Ahmar Suhail <ah...@apache.org> on 2025/08/07 20:35:43 UTC
[VOTE] Release Apache Hadoop 3.4.2
Hey all,
RC2 for Hadoop 3.4.2 is now available for voting.
There are a couple of things to note:
1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
working on the fix.
2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
For S3A, download SDK bundle v2.29.52 from:
https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
and drop it into /share/hadoop/common/lib. Release notes will be updated
with these instructions.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
The git tag is release-3.4.2-RC2, commit
e1c0dee881820a4d834ec4a4d2c70d0d953bb933
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1441
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
This release has been created off of branch-3.4.
Thanks,
Ahmar
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
Thanks Ahmar for your work on this. Will check it ASAP.
Best Regards,
- He Xiaoqiao
On Thu, Aug 14, 2025 at 5:14 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
Thanks Ahmar for your work on this. Will check it ASAP.
Best Regards,
- He Xiaoqiao
On Thu, Aug 14, 2025 at 5:14 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
sorry, been behind on this.
-1 packaging related.
that hadoop-3.4.2-lean.tar.gz file is actually a tar file. Took me a while
to work this down as I just assumed my ant build script was at fault.
file downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz
downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz: POSIX tar archive
I have verified that copying the right aws sdk version into tools/lib puts
it on the classpath.
Also found an NPE in the sequence new AzureBlobFileSystem().close() (i.e.
no Initialize)
Filed HADOOP-19650 for that
Also, there's no sha512 of the ant lean stuff.
Can't get
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
to downloads/incoming/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
Actual binaries, maven artifacts, good.
Parquet tests are failing with that buffer release logic which we have now
made optional in our code (HADOOP`18296). I'm wondering if we should have
parquet make vector read optional rather than on by default -but advise
turning on for all cloud stores.
Really unsure what to do there.
On Thu, 14 Aug 2025 at 10:13, Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
Thanks Ahmar for your work on this. Will check it ASAP.
Best Regards,
- He Xiaoqiao
On Thu, Aug 14, 2025 at 5:14 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
sorry, been behind on this.
-1 packaging related.
that hadoop-3.4.2-lean.tar.gz file is actually a tar file. Took me a while
to work this down as I just assumed my ant build script was at fault.
file downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz
downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz: POSIX tar archive
I have verified that copying the right aws sdk version into tools/lib puts
it on the classpath.
Also found an NPE in the sequence new AzureBlobFileSystem().close() (i.e.
no Initialize)
Filed HADOOP-19650 for that
Also, there's no sha512 of the ant lean stuff.
Can't get
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
to downloads/incoming/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
Actual binaries, maven artifacts, good.
Parquet tests are failing with that buffer release logic which we have now
made optional in our code (HADOOP`18296). I'm wondering if we should have
parquet make vector read optional rather than on by default -but advise
turning on for all cloud stores.
Really unsure what to do there.
On Thu, 14 Aug 2025 at 10:13, Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Dongjoon Hyun <do...@apache.org>.
+1 (non-binding)
I tested with Apache ORC 2.3.0-SNAPSHOT.
Thank you all!
Dongjoon Hyun
On 2025/08/26 11:56:31 Masatake Iwasaki wrote:
> +1 (binding)
>
> + verified checksums and signatures of src and site tarballs.
> + built from source tarball with native profile enabled on
> Rocky Linux 9 (x86_64 and aarch64).
> + deployed pseudo cluster with and without kerberos enabled.
> + ran some example jobs.
> + ran some simple query against httpfs.
> + skimmed the contents of pre-built site documentation.
>
> Thanks for the great work, Ahmar.
>
> Masatake Iwasaki
>
> On 8/21/2025 12:13 AM, Suhail, Ahmar wrote:
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >
> >
> >
> >
> >
> >
> >> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> >> Thanks Masatake and Steve,
> >>
> >> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
> >>
> >> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
> >>
> >> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> >>
> >> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
> >>
> >>
> >> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> >>
> >>
> >>
> >>
> >>
> >>
> >> Thanks for the update, Ahmar
> >>
> >>
> >> The binary artifacts basically looks good.
> >> I have some comments about inconsistencies.
> >>
> >>
> >> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
> >>
> >>
> >> Jira/Git commit message diff starting: ##############################################
> >> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> >> ...
> >> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> >> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> >> ...
> >> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> >>
> >>
> >> - The top page of site documentation was not updated for 3.4.2.
> >> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
> >>
> >>
> >> - Some release artifact are inconsistent as Steve pointed out.
> >>
> >>
> >> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> >>
> >>
> >> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> >>
> >>
> >> Masatake Iwasaki
> >>
> >>
> >> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> >>> Hey all,
> >>>
> >>> Just a reminder about this. Would be great to get some votes in asap and
> >>> hopefully finalise 3.4.2.
> >>>
> >>> Thanks,
> >>> Ahmar
> >>>
> >>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
> >>>
> >>>> Hey all,
> >>>>
> >>>> RC2 for Hadoop 3.4.2 is now available for voting.
> >>>>
> >>>> There are a couple of things to note:
> >>>>
> >>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
> >>>> working on the fix.
> >>>>
> >>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> >>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> >>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
> >>>> For S3A, download SDK bundle v2.29.52 from:
> >>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
> >>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
> >>>> with these instructions.
> >>>>
> >>>>
> >>>> The RC is available at:
> >>>>
> >>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> >>>>
> >>>> The git tag is release-3.4.2-RC2, commit
> >>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >>>>
> >>>> The maven artifacts are staged at:
> >>>>
> >>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> >>>>
> >>>>
> >>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >>>>
> >>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> >>>>
> >>>>
> >>>> This release has been created off of branch-3.4.
> >>>>
> >>>> Thanks,
> >>>> Ahmar
> >>>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
+1 (binding)
+ verified checksums and signatures of src and site tarballs.
+ built from source tarball with native profile enabled on
Rocky Linux 9 (x86_64 and aarch64).
+ deployed pseudo cluster with and without kerberos enabled.
+ ran some example jobs.
+ ran some simple query against httpfs.
+ skimmed the contents of pre-built site documentation.
Thanks for the great work, Ahmar.
Masatake Iwasaki
On 8/21/2025 12:13 AM, Suhail, Ahmar wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
>> Thanks Masatake and Steve,
>>
>> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>>
>> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>>
>> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>>
>>
>> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>>
>>
>>
>>
>>
>>
>> Thanks for the update, Ahmar
>>
>>
>> The binary artifacts basically looks good.
>> I have some comments about inconsistencies.
>>
>>
>> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>>
>>
>> Jira/Git commit message diff starting: ##############################################
>> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
>> ...
>> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
>> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
>> ...
>> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>>
>>
>> - The top page of site documentation was not updated for 3.4.2.
>> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>>
>>
>> - Some release artifact are inconsistent as Steve pointed out.
>>
>>
>> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>>
>>
>> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>>
>>
>> Masatake Iwasaki
>>
>>
>> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>>> Hey all,
>>>
>>> Just a reminder about this. Would be great to get some votes in asap and
>>> hopefully finalise 3.4.2.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>>
>>>> Hey all,
>>>>
>>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>>
>>>> There are a couple of things to note:
>>>>
>>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>>> working on the fix.
>>>>
>>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>>> For S3A, download SDK bundle v2.29.52 from:
>>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>>> with these instructions.
>>>>
>>>>
>>>> The RC is available at:
>>>>
>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>>
>>>> The git tag is release-3.4.2-RC2, commit
>>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>>
>>>> The maven artifacts are staged at:
>>>>
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>>
>>>>
>>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>>
>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>>
>>>>
>>>> This release has been created off of branch-3.4.
>>>>
>>>> Thanks,
>>>> Ahmar
>>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for testing and voting, and apologies for not creating a separate thread for RC3, will keep that in mind next time.
The voting summary is:
+5 binding (Ayush, Steve, He Xiaoqiao, Mukund, Masatake)
+3 non-binding (Erdogan, Dongjoon, Cheng Pan)
With 5 binding votes, the Hadoop release 3.4.2 has passed.
I will now process with the next steps to finalise it.
Thanks again to everyone for the help!
Ahmar
On 20/08/2025, 16:14, "Suhail, Ahmar" <ahmarsu@amazon.co.uk.INVA <ma...@amazon.co.uk.INVA>LID> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt;>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt;>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt;>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for voting,
Voting summary so far:
+4 binding (Ayush, Steve, He Xiaoqiao, Mukund)
+1 non-binding (Erdogan)
Looking for one more binding vote, and I will also add mine soon. Hoping to close voting by the end of the week and get the release out, so please do test and vote soon.
Thanks,
Ahmar
On 26/08/2025, 07:20, "Ayush Saxena" <ayushtkn@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <hexiaoqiao@apache.org <ma...@apache.org>> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mthakur@cloudera.com.inva <ma...@cloudera.com.inva>lid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <erdoganozkoca@gmail.com <ma...@gmail.com>>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556 <https://issues.apache.org/jira/browse/HADOOP-19556>
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com> <mailto:
> > > > > anujmodi2011@gmail.com <ma...@gmail.com>>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888> <
> > > > > https://github.com/apache/hadoop/pull/7888> <https://github.com/apache/hadoop/pull/7888>>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889> <
> > > > > https://github.com/apache/hadoop/pull/7889> <https://github.com/apache/hadoop/pull/7889>>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva <ma...@cloudera.com.inva>
> > > > > <mailto:stevel@cloudera.com.inva <ma...@cloudera.com.inva>>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>
> > > > > <mailto:ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443> <https://repository.apache.org/content/repositories/orgapachehadoop-1443>>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > https://github.com/apache/hadoop/pull/7887>> <https://github.com/apache/hadoop/pull/7887&gt;>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> <mailto:
> > > ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&amp;gt>
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&amp;gt;&gt>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt <https://repository.apache.org/content/repositories/orgapachehadoop-1441&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&amp;gt;&gt>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for voting,
Voting summary so far:
+4 binding (Ayush, Steve, He Xiaoqiao, Mukund)
+1 non-binding (Erdogan)
Looking for one more binding vote, and I will also add mine soon. Hoping to close voting by the end of the week and get the release out, so please do test and vote soon.
Thanks,
Ahmar
On 26/08/2025, 07:20, "Ayush Saxena" <ayushtkn@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <hexiaoqiao@apache.org <ma...@apache.org>> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mthakur@cloudera.com.inva <ma...@cloudera.com.inva>lid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <erdoganozkoca@gmail.com <ma...@gmail.com>>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556 <https://issues.apache.org/jira/browse/HADOOP-19556>
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com> <mailto:
> > > > > anujmodi2011@gmail.com <ma...@gmail.com>>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888> <
> > > > > https://github.com/apache/hadoop/pull/7888> <https://github.com/apache/hadoop/pull/7888>>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889> <
> > > > > https://github.com/apache/hadoop/pull/7889> <https://github.com/apache/hadoop/pull/7889>>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva <ma...@cloudera.com.inva>
> > > > > <mailto:stevel@cloudera.com.inva <ma...@cloudera.com.inva>>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>
> > > > > <mailto:ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443> <https://repository.apache.org/content/repositories/orgapachehadoop-1443>>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > https://github.com/apache/hadoop/pull/7887>> <https://github.com/apache/hadoop/pull/7887&gt;>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> <mailto:
> > > ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&amp;gt>
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&amp;gt;&gt>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt <https://repository.apache.org/content/repositories/orgapachehadoop-1441&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&amp;gt;&gt>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for voting,
Voting summary so far:
+4 binding (Ayush, Steve, He Xiaoqiao, Mukund)
+1 non-binding (Erdogan)
Looking for one more binding vote, and I will also add mine soon. Hoping to close voting by the end of the week and get the release out, so please do test and vote soon.
Thanks,
Ahmar
On 26/08/2025, 07:20, "Ayush Saxena" <ayushtkn@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <hexiaoqiao@apache.org <ma...@apache.org>> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mthakur@cloudera.com.inva <ma...@cloudera.com.inva>lid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <erdoganozkoca@gmail.com <ma...@gmail.com>>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556 <https://issues.apache.org/jira/browse/HADOOP-19556>
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com> <mailto:
> > > > > anujmodi2011@gmail.com <ma...@gmail.com>>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888> <
> > > > > https://github.com/apache/hadoop/pull/7888> <https://github.com/apache/hadoop/pull/7888>>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889> <
> > > > > https://github.com/apache/hadoop/pull/7889> <https://github.com/apache/hadoop/pull/7889>>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva <ma...@cloudera.com.inva>
> > > > > <mailto:stevel@cloudera.com.inva <ma...@cloudera.com.inva>>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>
> > > > > <mailto:ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443> <https://repository.apache.org/content/repositories/orgapachehadoop-1443>>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > https://github.com/apache/hadoop/pull/7887>> <https://github.com/apache/hadoop/pull/7887&gt;>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> <mailto:
> > > ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&amp;gt>
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&amp;gt;&gt>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt <https://repository.apache.org/content/repositories/orgapachehadoop-1441&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&amp;gt;&gt>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for voting,
Voting summary so far:
+4 binding (Ayush, Steve, He Xiaoqiao, Mukund)
+1 non-binding (Erdogan)
Looking for one more binding vote, and I will also add mine soon. Hoping to close voting by the end of the week and get the release out, so please do test and vote soon.
Thanks,
Ahmar
On 26/08/2025, 07:20, "Ayush Saxena" <ayushtkn@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <hexiaoqiao@apache.org <ma...@apache.org>> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mthakur@cloudera.com.inva <ma...@cloudera.com.inva>lid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <erdoganozkoca@gmail.com <ma...@gmail.com>>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556 <https://issues.apache.org/jira/browse/HADOOP-19556>
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com> <mailto:
> > > > > anujmodi2011@gmail.com <ma...@gmail.com>>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888> <
> > > > > https://github.com/apache/hadoop/pull/7888> <https://github.com/apache/hadoop/pull/7888>>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889> <
> > > > > https://github.com/apache/hadoop/pull/7889> <https://github.com/apache/hadoop/pull/7889>>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva <ma...@cloudera.com.inva>
> > > > > <mailto:stevel@cloudera.com.inva <ma...@cloudera.com.inva>>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>
> > > > > <mailto:ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443> <https://repository.apache.org/content/repositories/orgapachehadoop-1443>>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> <
> > > > > https://github.com/apache/hadoop/pull/7887>> <https://github.com/apache/hadoop/pull/7887&gt;>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:
> > > iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> <mailto:
> > > ahmar@apache.org <ma...@apache.org>
> > > > > <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&amp;gt>
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&amp;gt;&gt>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt <https://repository.apache.org/content/repositories/orgapachehadoop-1441&amp;gt>
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&amp;gt;&gt>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > <mailto:common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ayush Saxena <ay...@gmail.com>.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <he...@apache.org> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mt...@cloudera.com.invalid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > > anujmodi2011@gmail.com>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > > https://github.com/apache/hadoop/pull/7888>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > > https://github.com/apache/hadoop/pull/7889>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva
> > > > > <ma...@cloudera.com.inva>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva
> > > > > <ma...@amazon.co.uk.inva>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > > <ma...@apache.org>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > > ahmar@apache.org
> > > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > > ahmar@apache.org>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ayush Saxena <ay...@gmail.com>.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <he...@apache.org> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mt...@cloudera.com.invalid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > > anujmodi2011@gmail.com>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > > https://github.com/apache/hadoop/pull/7888>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > > https://github.com/apache/hadoop/pull/7889>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva
> > > > > <ma...@cloudera.com.inva>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva
> > > > > <ma...@amazon.co.uk.inva>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > > <ma...@apache.org>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > > ahmar@apache.org
> > > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > > ahmar@apache.org>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ayush Saxena <ay...@gmail.com>.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <he...@apache.org> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mt...@cloudera.com.invalid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > > anujmodi2011@gmail.com>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > > https://github.com/apache/hadoop/pull/7888>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > > https://github.com/apache/hadoop/pull/7889>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva
> > > > > <ma...@cloudera.com.inva>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva
> > > > > <ma...@amazon.co.uk.inva>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > > <ma...@apache.org>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > > ahmar@apache.org
> > > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > > ahmar@apache.org>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ayush Saxena <ay...@gmail.com>.
+1 (Binding)
* Built from source (X86 & ARM)
* Validated checksums.
* Verified Signatures.
* Verified all source files have ASF headers
* Validated NOTICE & LICENSE files
* Verified no diff b/w git tag & src tar
* Verified the output of `hadoop version`
* Ran some basic HDFS commands
* Ran example jobs: teragen, terasort, teravalidate, pi
* Browsed through the UI: NN, DN, RM, NM & JHS
* Skimmed over the contents of site tar
Thanx Ahmar for driving the release, Good Luck!!!
-Ayush
On Tue, 26 Aug 2025 at 08:01, Xiaoqiao He <he...@apache.org> wrote:
>
> +1(binding).
>
> [Y] Verified sha512 checksum and signature.
> [Y] LICENSE files exist and NOTICE is included in source tarball.
> [Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
> -DskipTests -Pnative -Pdist -Dtar`.
> [Y] Rat check is ok. `mvn clean apache-rat:check`
> [Y] Setup pseudo cluster with HDFS and YARN.
> [Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
> [Y] Run example mr jobs and check the result - Pi & wordcount.
> [Y] Spot-check and run some unit tests.
> [Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
> [Y] Skimmed over the contents of site documentation.
> [Y] Skimmed over the contents of maven repo.
>
> Thanks Ahmar for your work.
>
> Best Regards,
> - He Xiaoqiao
>
> On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
> <mt...@cloudera.com.invalid> wrote:
>
> > +1 binding
> > Verified signature and checksums.
> > Compiled the full source tree successfully.
> > Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> > on us-west2 bucket which verified dropping in of aws sdk library.
> > Also verified access to abfs containers.
> > Compiled gcs connector using this release.
> >
> > Thanks Ahmar for driving the release.
> >
> > PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> > the norm is to create a new email for every RC vote as per my
> > understanding.
> >
> > On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> > >
> > wrote:
> >
> > > +1 binding
> > >
> > > * downloaded, checked signing and hashes of binary, arm binary and source
> > > artifacts
> > > * full clean build of source
> > > * aws test runs against s3 london and dell ecs. One oddity in building
> > from
> > > the untarred source tree, Terasort failing with history server not
> > > starting. This did work in my git copy of the source so I'm assuming I've
> > > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > > the cluster tests run, which is an improvement over trunk right now
> > > * full s3a command line testing as per my qualifying and sdk doc.
> > >
> > >
> > > Steve
> > >
> > >
> > >
> > >
> > >
> > > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > * Verified sha512 checksum was correct for source tarball
> > > > * Verified signature was correct for source tarball (not verified
> > trust)
> > > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > > non-blocker for release as
> > > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > > passing against Amazon S3 in eu-west-1
> > > >
> > > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > > tarihinde şunu yazdı:
> > > >
> > > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > > included in the release yet, if the RC fails the vote, will include
> > it
> > > in
> > > > > the next one.
> > > > >
> > > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > > anujmodi2011@gmail.com>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi, Ahmar
> > > > >
> > > > >
> > > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> > raised
> > > > on
> > > > > RC.
> > > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > > https://github.com/apache/hadoop/pull/7888>
> > > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > > https://github.com/apache/hadoop/pull/7889>
> > > > >
> > > > >
> > > > > Feel free to merge them as per the release timelines.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > > <stevel@cloudera.com.inva
> > > > > <ma...@cloudera.com.inva>lid>
> > > > > wrote:
> > > > >
> > > > >
> > > > > > just discovering as I do the third party compatibility testing of
> > the
> > > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > > broken
> > > > > > -nothing to worry about though as its a new feature so not a
> > > regression
> > > > > >
> > > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > > <ahmarsu@amazon.co.uk.inva
> > > > > <ma...@amazon.co.uk.inva>lid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > I've fixed the following issues:
> > > > > > > 1) Issues with changelog, release notes and landing page.
> > Created a
> > > > new
> > > > > > > build so they now include all the JIRA's that were missing
> > before.
> > > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> > Ignoring
> > > > > > > unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > message when untarring
> > > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > > >
> > > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > > >
> > > > > > > The RC is available at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > > >
> > > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > > >
> > > > > > > The maven artifacts are staged at:
> > > > > > >
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > > >
> > > > > > > You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > > >
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > > >
> > > > > > >
> > > > > > > This release has been created off of branch-3.4.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ahmar
> > > > > > >
> > > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>
> > > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the organization.
> > Do
> > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > >
> > > > > > >
> > > > > > > I think just running the create-release[1] script could be an
> > easy
> > > > way
> > > > > > > while it does not provide options for building only site or
> > source
> > > > > > > tarballs (now).
> > > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > > RELEASENOTES,
> > > > > > > it would be nice to update the binary tarballs.
> > > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > > tarballs
> > > > > > > # could be ok if only documentation is updated.
> > > > > > > # Signing and checksums must be cared in that case.
> > > > > > >
> > > > > > >
> > > > > > > In addition, I got following warnings when I extracted the binary
> > > > > > tarballs
> > > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> > on
> > > > > > > Ubuntu.::
> > > > > > >
> > > > > > >
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > > tar: Ignoring unknown extended header keyword
> > > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > >
> > > > > > >
> > > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > > It would be nice to fix too.
> > > > > > >
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > > Thanks Masatake and Steve,
> > > > > > > >
> > > > > > > > Here's the PR for updating the top page of the site
> > > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> > anything
> > > > else
> > > > > > we
> > > > > > > think needs to go in there, let me know, otherwise I will merge
> > by
> > > > > > tomorrow.
> > > > > > > >
> > > > > > > > Since I have to regenerate the site and changelog, do I need to
> > > do
> > > > > the
> > > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > > maybe?)
> > > > > > > >
> > > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> > and
> > > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > > >
> > > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > > iwasakims@oss.nttdata.com
> > > > > <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>
> > > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > > iwasakims@oss.nttdata.com
> > > > >>>>
> > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > CAUTION: This email originated from outside of the
> > organization.
> > > Do
> > > > > not
> > > > > > > click links or open attachments unless you can confirm the sender
> > > and
> > > > > > know
> > > > > > > the content is safe.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks for the update, Ahmar
> > > > > > > >
> > > > > > > >
> > > > > > > > The binary artifacts basically looks good.
> > > > > > > > I have some comments about inconsistencies.
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > > Version/s". I updated them.
> > > > > > > >
> > > > > > > >
> > > > > > > > Jira/Git commit message diff starting:
> > > > > > > ##############################################
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> > SSE-KMS
> > > > > #7816
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> > Updates
> > > > AAL
> > > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > > LocalDirAllocator
> > > > > > > still doesn't always recover from directory deletion (#7683)
> > > (#7685)
> > > > > > > > ...
> > > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > > commons-cli
> > > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > > >
> > > > > > > >
> > > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > > >
> > > > > > >
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > > <
> > > > >
> > > >
> > >
> > https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >
> > > > > > > >
> > > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > > >
> > > > > > > >
> > > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > > >
> > > > > > > >
> > > > > > > > Masatake Iwasaki
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > > >> Hey all,
> > > > > > > >>
> > > > > > > >> Just a reminder about this. Would be great to get some votes
> > in
> > > > asap
> > > > > > and
> > > > > > > >> hopefully finalise 3.4.2.
> > > > > > > >>
> > > > > > > >> Thanks,
> > > > > > > >> Ahmar
> > > > > > > >>
> > > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > > <ma...@apache.org>
> > > > > > <mailto:
> > > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > > ahmar@apache.org
> > > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > > ahmar@apache.org>>>>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >>> Hey all,
> > > > > > > >>>
> > > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > > >>>
> > > > > > > >>> There are a couple of things to note:
> > > > > > > >>>
> > > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> > Wei-Chiu
> > > > > > Chuang
> > > > > > > for
> > > > > > > >>> working on the fix.
> > > > > > > >>>
> > > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > > bundle
> > > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> > now
> > > > > > ~600MB,
> > > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > > uploaded
> > > > > > to
> > > > > > > SVN.
> > > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > <
> > > > > > >
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > <
> > > > >
> > > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > > >
> > > > > > >
> > > > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > ;>
> > > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > > <
> > > > >
> > > >
> > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > > >
> > > > > > > ;>,
> > > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > > be
> > > > > > > updated
> > > > > > > >>> with these instructions.
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> The RC is available at:
> > > > > > > >>>
> > > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> > ;>
> > > > > > > >>>
> > > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > > >>>
> > > > > > > >>> The maven artifacts are staged at:
> > > > > > > >>>
> > > > > > > >>>
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > > <
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > >
> > > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > ;>
> > > > > > <
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > >
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > > >
> > > > > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> You can find my public key
> > > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > > at:
> > > > > > > >>>
> > > > > > > >>>
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > > <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > ;>
> > > <
> > > > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > > ;>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> This release has been created off of branch-3.4.
> > > > > > > >>>
> > > > > > > >>> Thanks,
> > > > > > > >>> Ahmar
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > > For additional commands, e-mail:
> > common-dev-help@hadoop.apache.org
> > > > > <ma...@hadoop.apache.org>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
+1(binding).
[Y] Verified sha512 checksum and signature.
[Y] LICENSE files exist and NOTICE is included in source tarball.
[Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
-DskipTests -Pnative -Pdist -Dtar`.
[Y] Rat check is ok. `mvn clean apache-rat:check`
[Y] Setup pseudo cluster with HDFS and YARN.
[Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
[Y] Run example mr jobs and check the result - Pi & wordcount.
[Y] Spot-check and run some unit tests.
[Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
[Y] Skimmed over the contents of site documentation.
[Y] Skimmed over the contents of maven repo.
Thanks Ahmar for your work.
Best Regards,
- He Xiaoqiao
On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
<mt...@cloudera.com.invalid> wrote:
> +1 binding
> Verified signature and checksums.
> Compiled the full source tree successfully.
> Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> on us-west2 bucket which verified dropping in of aws sdk library.
> Also verified access to abfs containers.
> Compiled gcs connector using this release.
>
> Thanks Ahmar for driving the release.
>
> PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> the norm is to create a new email for every RC vote as per my
> understanding.
>
> On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > +1 binding
> >
> > * downloaded, checked signing and hashes of binary, arm binary and source
> > artifacts
> > * full clean build of source
> > * aws test runs against s3 london and dell ecs. One oddity in building
> from
> > the untarred source tree, Terasort failing with history server not
> > starting. This did work in my git copy of the source so I'm assuming I've
> > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > the cluster tests run, which is an improvement over trunk right now
> > * full s3a command line testing as per my qualifying and sdk doc.
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > * Verified sha512 checksum was correct for source tarball
> > > * Verified signature was correct for source tarball (not verified
> trust)
> > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > non-blocker for release as
> > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > passing against Amazon S3 in eu-west-1
> > >
> > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > tarihinde şunu yazdı:
> > >
> > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > included in the release yet, if the RC fails the vote, will include
> it
> > in
> > > > the next one.
> > > >
> > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > anujmodi2011@gmail.com>> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi, Ahmar
> > > >
> > > >
> > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> raised
> > > on
> > > > RC.
> > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > https://github.com/apache/hadoop/pull/7888>
> > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > https://github.com/apache/hadoop/pull/7889>
> > > >
> > > >
> > > > Feel free to merge them as per the release timelines.
> > > >
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > <stevel@cloudera.com.inva
> > > > <ma...@cloudera.com.inva>lid>
> > > > wrote:
> > > >
> > > >
> > > > > just discovering as I do the third party compatibility testing of
> the
> > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > broken
> > > > > -nothing to worry about though as its a new feature so not a
> > regression
> > > > >
> > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > <ahmarsu@amazon.co.uk.inva
> > > > <ma...@amazon.co.uk.inva>lid>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > I've fixed the following issues:
> > > > > > 1) Issues with changelog, release notes and landing page.
> Created a
> > > new
> > > > > > build so they now include all the JIRA's that were missing
> before.
> > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> Ignoring
> > > > > > unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > message when untarring
> > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > >
> > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > >
> > > > > > The RC is available at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > >
> > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > >
> > > > > > The maven artifacts are staged at:
> > > > > >
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > >
> > > > > > You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > >
> > > > > >
> > > > > > This release has been created off of branch-3.4.
> > > > > >
> > > > > > Thanks,
> > > > > > Ahmar
> > > > > >
> > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > >
> > > > > >
> > > > > > I think just running the create-release[1] script could be an
> easy
> > > way
> > > > > > while it does not provide options for building only site or
> source
> > > > > > tarballs (now).
> > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > RELEASENOTES,
> > > > > > it would be nice to update the binary tarballs.
> > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > tarballs
> > > > > > # could be ok if only documentation is updated.
> > > > > > # Signing and checksums must be cared in that case.
> > > > > >
> > > > > >
> > > > > > In addition, I got following warnings when I extracted the binary
> > > > > tarballs
> > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> on
> > > > > > Ubuntu.::
> > > > > >
> > > > > >
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > >
> > > > > >
> > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > It would be nice to fix too.
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > Thanks Masatake and Steve,
> > > > > > >
> > > > > > > Here's the PR for updating the top page of the site
> > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> anything
> > > else
> > > > > we
> > > > > > think needs to go in there, let me know, otherwise I will merge
> by
> > > > > tomorrow.
> > > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > > >
> > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> and
> > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > >
> > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>
> > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>>
> > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the
> organization.
> > Do
> > > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks for the update, Ahmar
> > > > > > >
> > > > > > >
> > > > > > > The binary artifacts basically looks good.
> > > > > > > I have some comments about inconsistencies.
> > > > > > >
> > > > > > >
> > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > Version/s". I updated them.
> > > > > > >
> > > > > > >
> > > > > > > Jira/Git commit message diff starting:
> > > > > > ##############################################
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> SSE-KMS
> > > > #7816
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> Updates
> > > AAL
> > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > LocalDirAllocator
> > > > > > still doesn't always recover from directory deletion (#7683)
> > (#7685)
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > commons-cli
> > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > >
> > > > > > >
> > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > >
> > > > > > ;>
> > > > > > >
> > > > > > >
> > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > >> Hey all,
> > > > > > >>
> > > > > > >> Just a reminder about this. Would be great to get some votes
> in
> > > asap
> > > > > and
> > > > > > >> hopefully finalise 3.4.2.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Ahmar
> > > > > > >>
> > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > <ma...@apache.org>
> > > > > <mailto:
> > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > ahmar@apache.org
> > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > ahmar@apache.org>>>>
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Hey all,
> > > > > > >>>
> > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > >>>
> > > > > > >>> There are a couple of things to note:
> > > > > > >>>
> > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> Wei-Chiu
> > > > > Chuang
> > > > > > for
> > > > > > >>> working on the fix.
> > > > > > >>>
> > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > bundle
> > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> now
> > > > > ~600MB,
> > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > uploaded
> > > > > to
> > > > > > SVN.
> > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > >>>
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > >
> > > > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > ;>
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > >
> > > > > > ;>,
> > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > be
> > > > > > updated
> > > > > > >>> with these instructions.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> The RC is available at:
> > > > > > >>>
> > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> ;>
> > > > > > >>>
> > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > >>>
> > > > > > >>> The maven artifacts are staged at:
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > >
> > > > > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> You can find my public key
> > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > at:
> > > > > > >>>
> > > > > > >>>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> ;>
> > <
> > > >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> This release has been created off of branch-3.4.
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Ahmar
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > > For additional commands, e-mail:
> common-dev-help@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
+1(binding).
[Y] Verified sha512 checksum and signature.
[Y] LICENSE files exist and NOTICE is included in source tarball.
[Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
-DskipTests -Pnative -Pdist -Dtar`.
[Y] Rat check is ok. `mvn clean apache-rat:check`
[Y] Setup pseudo cluster with HDFS and YARN.
[Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
[Y] Run example mr jobs and check the result - Pi & wordcount.
[Y] Spot-check and run some unit tests.
[Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
[Y] Skimmed over the contents of site documentation.
[Y] Skimmed over the contents of maven repo.
Thanks Ahmar for your work.
Best Regards,
- He Xiaoqiao
On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
<mt...@cloudera.com.invalid> wrote:
> +1 binding
> Verified signature and checksums.
> Compiled the full source tree successfully.
> Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> on us-west2 bucket which verified dropping in of aws sdk library.
> Also verified access to abfs containers.
> Compiled gcs connector using this release.
>
> Thanks Ahmar for driving the release.
>
> PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> the norm is to create a new email for every RC vote as per my
> understanding.
>
> On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > +1 binding
> >
> > * downloaded, checked signing and hashes of binary, arm binary and source
> > artifacts
> > * full clean build of source
> > * aws test runs against s3 london and dell ecs. One oddity in building
> from
> > the untarred source tree, Terasort failing with history server not
> > starting. This did work in my git copy of the source so I'm assuming I've
> > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > the cluster tests run, which is an improvement over trunk right now
> > * full s3a command line testing as per my qualifying and sdk doc.
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > * Verified sha512 checksum was correct for source tarball
> > > * Verified signature was correct for source tarball (not verified
> trust)
> > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > non-blocker for release as
> > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > passing against Amazon S3 in eu-west-1
> > >
> > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > tarihinde şunu yazdı:
> > >
> > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > included in the release yet, if the RC fails the vote, will include
> it
> > in
> > > > the next one.
> > > >
> > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > anujmodi2011@gmail.com>> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi, Ahmar
> > > >
> > > >
> > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> raised
> > > on
> > > > RC.
> > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > https://github.com/apache/hadoop/pull/7888>
> > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > https://github.com/apache/hadoop/pull/7889>
> > > >
> > > >
> > > > Feel free to merge them as per the release timelines.
> > > >
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > <stevel@cloudera.com.inva
> > > > <ma...@cloudera.com.inva>lid>
> > > > wrote:
> > > >
> > > >
> > > > > just discovering as I do the third party compatibility testing of
> the
> > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > broken
> > > > > -nothing to worry about though as its a new feature so not a
> > regression
> > > > >
> > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > <ahmarsu@amazon.co.uk.inva
> > > > <ma...@amazon.co.uk.inva>lid>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > I've fixed the following issues:
> > > > > > 1) Issues with changelog, release notes and landing page.
> Created a
> > > new
> > > > > > build so they now include all the JIRA's that were missing
> before.
> > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> Ignoring
> > > > > > unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > message when untarring
> > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > >
> > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > >
> > > > > > The RC is available at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > >
> > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > >
> > > > > > The maven artifacts are staged at:
> > > > > >
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > >
> > > > > > You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > >
> > > > > >
> > > > > > This release has been created off of branch-3.4.
> > > > > >
> > > > > > Thanks,
> > > > > > Ahmar
> > > > > >
> > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > >
> > > > > >
> > > > > > I think just running the create-release[1] script could be an
> easy
> > > way
> > > > > > while it does not provide options for building only site or
> source
> > > > > > tarballs (now).
> > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > RELEASENOTES,
> > > > > > it would be nice to update the binary tarballs.
> > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > tarballs
> > > > > > # could be ok if only documentation is updated.
> > > > > > # Signing and checksums must be cared in that case.
> > > > > >
> > > > > >
> > > > > > In addition, I got following warnings when I extracted the binary
> > > > > tarballs
> > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> on
> > > > > > Ubuntu.::
> > > > > >
> > > > > >
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > >
> > > > > >
> > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > It would be nice to fix too.
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > Thanks Masatake and Steve,
> > > > > > >
> > > > > > > Here's the PR for updating the top page of the site
> > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> anything
> > > else
> > > > > we
> > > > > > think needs to go in there, let me know, otherwise I will merge
> by
> > > > > tomorrow.
> > > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > > >
> > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> and
> > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > >
> > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>
> > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>>
> > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the
> organization.
> > Do
> > > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks for the update, Ahmar
> > > > > > >
> > > > > > >
> > > > > > > The binary artifacts basically looks good.
> > > > > > > I have some comments about inconsistencies.
> > > > > > >
> > > > > > >
> > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > Version/s". I updated them.
> > > > > > >
> > > > > > >
> > > > > > > Jira/Git commit message diff starting:
> > > > > > ##############################################
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> SSE-KMS
> > > > #7816
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> Updates
> > > AAL
> > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > LocalDirAllocator
> > > > > > still doesn't always recover from directory deletion (#7683)
> > (#7685)
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > commons-cli
> > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > >
> > > > > > >
> > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > >
> > > > > > ;>
> > > > > > >
> > > > > > >
> > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > >> Hey all,
> > > > > > >>
> > > > > > >> Just a reminder about this. Would be great to get some votes
> in
> > > asap
> > > > > and
> > > > > > >> hopefully finalise 3.4.2.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Ahmar
> > > > > > >>
> > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > <ma...@apache.org>
> > > > > <mailto:
> > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > ahmar@apache.org
> > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > ahmar@apache.org>>>>
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Hey all,
> > > > > > >>>
> > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > >>>
> > > > > > >>> There are a couple of things to note:
> > > > > > >>>
> > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> Wei-Chiu
> > > > > Chuang
> > > > > > for
> > > > > > >>> working on the fix.
> > > > > > >>>
> > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > bundle
> > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> now
> > > > > ~600MB,
> > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > uploaded
> > > > > to
> > > > > > SVN.
> > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > >>>
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > >
> > > > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > ;>
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > >
> > > > > > ;>,
> > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > be
> > > > > > updated
> > > > > > >>> with these instructions.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> The RC is available at:
> > > > > > >>>
> > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> ;>
> > > > > > >>>
> > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > >>>
> > > > > > >>> The maven artifacts are staged at:
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > >
> > > > > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> You can find my public key
> > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > at:
> > > > > > >>>
> > > > > > >>>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> ;>
> > <
> > > >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> This release has been created off of branch-3.4.
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Ahmar
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > > For additional commands, e-mail:
> common-dev-help@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
+1(binding).
[Y] Verified sha512 checksum and signature.
[Y] LICENSE files exist and NOTICE is included in source tarball.
[Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
-DskipTests -Pnative -Pdist -Dtar`.
[Y] Rat check is ok. `mvn clean apache-rat:check`
[Y] Setup pseudo cluster with HDFS and YARN.
[Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
[Y] Run example mr jobs and check the result - Pi & wordcount.
[Y] Spot-check and run some unit tests.
[Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
[Y] Skimmed over the contents of site documentation.
[Y] Skimmed over the contents of maven repo.
Thanks Ahmar for your work.
Best Regards,
- He Xiaoqiao
On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
<mt...@cloudera.com.invalid> wrote:
> +1 binding
> Verified signature and checksums.
> Compiled the full source tree successfully.
> Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> on us-west2 bucket which verified dropping in of aws sdk library.
> Also verified access to abfs containers.
> Compiled gcs connector using this release.
>
> Thanks Ahmar for driving the release.
>
> PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> the norm is to create a new email for every RC vote as per my
> understanding.
>
> On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > +1 binding
> >
> > * downloaded, checked signing and hashes of binary, arm binary and source
> > artifacts
> > * full clean build of source
> > * aws test runs against s3 london and dell ecs. One oddity in building
> from
> > the untarred source tree, Terasort failing with history server not
> > starting. This did work in my git copy of the source so I'm assuming I've
> > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > the cluster tests run, which is an improvement over trunk right now
> > * full s3a command line testing as per my qualifying and sdk doc.
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > * Verified sha512 checksum was correct for source tarball
> > > * Verified signature was correct for source tarball (not verified
> trust)
> > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > non-blocker for release as
> > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > passing against Amazon S3 in eu-west-1
> > >
> > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > tarihinde şunu yazdı:
> > >
> > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > included in the release yet, if the RC fails the vote, will include
> it
> > in
> > > > the next one.
> > > >
> > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > anujmodi2011@gmail.com>> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi, Ahmar
> > > >
> > > >
> > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> raised
> > > on
> > > > RC.
> > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > https://github.com/apache/hadoop/pull/7888>
> > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > https://github.com/apache/hadoop/pull/7889>
> > > >
> > > >
> > > > Feel free to merge them as per the release timelines.
> > > >
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > <stevel@cloudera.com.inva
> > > > <ma...@cloudera.com.inva>lid>
> > > > wrote:
> > > >
> > > >
> > > > > just discovering as I do the third party compatibility testing of
> the
> > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > broken
> > > > > -nothing to worry about though as its a new feature so not a
> > regression
> > > > >
> > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > <ahmarsu@amazon.co.uk.inva
> > > > <ma...@amazon.co.uk.inva>lid>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > I've fixed the following issues:
> > > > > > 1) Issues with changelog, release notes and landing page.
> Created a
> > > new
> > > > > > build so they now include all the JIRA's that were missing
> before.
> > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> Ignoring
> > > > > > unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > message when untarring
> > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > >
> > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > >
> > > > > > The RC is available at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > >
> > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > >
> > > > > > The maven artifacts are staged at:
> > > > > >
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > >
> > > > > > You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > >
> > > > > >
> > > > > > This release has been created off of branch-3.4.
> > > > > >
> > > > > > Thanks,
> > > > > > Ahmar
> > > > > >
> > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > >
> > > > > >
> > > > > > I think just running the create-release[1] script could be an
> easy
> > > way
> > > > > > while it does not provide options for building only site or
> source
> > > > > > tarballs (now).
> > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > RELEASENOTES,
> > > > > > it would be nice to update the binary tarballs.
> > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > tarballs
> > > > > > # could be ok if only documentation is updated.
> > > > > > # Signing and checksums must be cared in that case.
> > > > > >
> > > > > >
> > > > > > In addition, I got following warnings when I extracted the binary
> > > > > tarballs
> > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> on
> > > > > > Ubuntu.::
> > > > > >
> > > > > >
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > >
> > > > > >
> > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > It would be nice to fix too.
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > Thanks Masatake and Steve,
> > > > > > >
> > > > > > > Here's the PR for updating the top page of the site
> > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> anything
> > > else
> > > > > we
> > > > > > think needs to go in there, let me know, otherwise I will merge
> by
> > > > > tomorrow.
> > > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > > >
> > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> and
> > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > >
> > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>
> > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>>
> > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the
> organization.
> > Do
> > > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks for the update, Ahmar
> > > > > > >
> > > > > > >
> > > > > > > The binary artifacts basically looks good.
> > > > > > > I have some comments about inconsistencies.
> > > > > > >
> > > > > > >
> > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > Version/s". I updated them.
> > > > > > >
> > > > > > >
> > > > > > > Jira/Git commit message diff starting:
> > > > > > ##############################################
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> SSE-KMS
> > > > #7816
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> Updates
> > > AAL
> > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > LocalDirAllocator
> > > > > > still doesn't always recover from directory deletion (#7683)
> > (#7685)
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > commons-cli
> > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > >
> > > > > > >
> > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > >
> > > > > > ;>
> > > > > > >
> > > > > > >
> > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > >> Hey all,
> > > > > > >>
> > > > > > >> Just a reminder about this. Would be great to get some votes
> in
> > > asap
> > > > > and
> > > > > > >> hopefully finalise 3.4.2.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Ahmar
> > > > > > >>
> > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > <ma...@apache.org>
> > > > > <mailto:
> > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > ahmar@apache.org
> > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > ahmar@apache.org>>>>
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Hey all,
> > > > > > >>>
> > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > >>>
> > > > > > >>> There are a couple of things to note:
> > > > > > >>>
> > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> Wei-Chiu
> > > > > Chuang
> > > > > > for
> > > > > > >>> working on the fix.
> > > > > > >>>
> > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > bundle
> > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> now
> > > > > ~600MB,
> > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > uploaded
> > > > > to
> > > > > > SVN.
> > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > >>>
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > >
> > > > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > ;>
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > >
> > > > > > ;>,
> > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > be
> > > > > > updated
> > > > > > >>> with these instructions.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> The RC is available at:
> > > > > > >>>
> > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> ;>
> > > > > > >>>
> > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > >>>
> > > > > > >>> The maven artifacts are staged at:
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > >
> > > > > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> You can find my public key
> > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > at:
> > > > > > >>>
> > > > > > >>>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> ;>
> > <
> > > >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> This release has been created off of branch-3.4.
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Ahmar
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > > For additional commands, e-mail:
> common-dev-help@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
+1(binding).
[Y] Verified sha512 checksum and signature.
[Y] LICENSE files exist and NOTICE is included in source tarball.
[Y] Build the source code on Ubuntu and OpenJDK 11 by `mvn clean package
-DskipTests -Pnative -Pdist -Dtar`.
[Y] Rat check is ok. `mvn clean apache-rat:check`
[Y] Setup pseudo cluster with HDFS and YARN.
[Y] Run simple FsShell - mkdir/put/get/mv/rm and check the result.
[Y] Run example mr jobs and check the result - Pi & wordcount.
[Y] Spot-check and run some unit tests.
[Y] Skimmed the Web UI of NameNode/DataNode/Resourcemanager/NodeManager.
[Y] Skimmed over the contents of site documentation.
[Y] Skimmed over the contents of maven repo.
Thanks Ahmar for your work.
Best Regards,
- He Xiaoqiao
On Tue, Aug 26, 2025 at 8:09 AM Mukund Madhav Thakur
<mt...@cloudera.com.invalid> wrote:
> +1 binding
> Verified signature and checksums.
> Compiled the full source tree successfully.
> Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
> on us-west2 bucket which verified dropping in of aws sdk library.
> Also verified access to abfs containers.
> Compiled gcs connector using this release.
>
> Thanks Ahmar for driving the release.
>
> PS: I was expecting a new email for RC3 as RC2 had some problems. I think
> the norm is to create a new email for every RC vote as per my
> understanding.
>
> On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > +1 binding
> >
> > * downloaded, checked signing and hashes of binary, arm binary and source
> > artifacts
> > * full clean build of source
> > * aws test runs against s3 london and dell ecs. One oddity in building
> from
> > the untarred source tree, Terasort failing with history server not
> > starting. This did work in my git copy of the source so I'm assuming I've
> > got the core-site/auth-keys wrong (the ones which get copied in). Still,
> > the cluster tests run, which is an improvement over trunk right now
> > * full s3a command line testing as per my qualifying and sdk doc.
> >
> >
> > Steve
> >
> >
> >
> >
> >
> > On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > * Verified sha512 checksum was correct for source tarball
> > > * Verified signature was correct for source tarball (not verified
> trust)
> > > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > > non-blocker for release as
> > > https://issues.apache.org/jira/browse/HADOOP-19556
> > > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > > passing against Amazon S3 in eu-west-1
> > >
> > > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > > tarihinde şunu yazdı:
> > >
> > > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > > included in the release yet, if the RC fails the vote, will include
> it
> > in
> > > > the next one.
> > > >
> > > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > > anujmodi2011@gmail.com>> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi, Ahmar
> > > >
> > > >
> > > > Here are the PRs for the fix raised for ABFS close() causing NPE
> raised
> > > on
> > > > RC.
> > > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > > https://github.com/apache/hadoop/pull/7888>
> > > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > > https://github.com/apache/hadoop/pull/7889>
> > > >
> > > >
> > > > Feel free to merge them as per the release timelines.
> > > >
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> > <stevel@cloudera.com.inva
> > > > <ma...@cloudera.com.inva>lid>
> > > > wrote:
> > > >
> > > >
> > > > > just discovering as I do the third party compatibility testing of
> the
> > > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > > broken
> > > > > -nothing to worry about though as its a new feature so not a
> > regression
> > > > >
> > > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> > <ahmarsu@amazon.co.uk.inva
> > > > <ma...@amazon.co.uk.inva>lid>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > I've fixed the following issues:
> > > > > > 1) Issues with changelog, release notes and landing page.
> Created a
> > > new
> > > > > > build so they now include all the JIRA's that were missing
> before.
> > > > > > 2) Create the lean tar on ubuntu so you won't get the tar:
> Ignoring
> > > > > > unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > message when untarring
> > > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > > >
> > > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > > >
> > > > > > The RC is available at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > > >
> > > > > > The git tag is release-3.4.2-RC3, commit
> > > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > > >
> > > > > > The maven artifacts are staged at:
> > > > > >
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > > >
> > > > > > You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > > >
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > > >
> > > > > >
> > > > > > This release has been created off of branch-3.4.
> > > > > >
> > > > > > Thanks,
> > > > > > Ahmar
> > > > > >
> > > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>
> > > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > >
> > > > > >
> > > > > > I think just running the create-release[1] script could be an
> easy
> > > way
> > > > > > while it does not provide options for building only site or
> source
> > > > > > tarballs (now).
> > > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > > RELEASENOTES,
> > > > > > it would be nice to update the binary tarballs.
> > > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> > tarballs
> > > > > > # could be ok if only documentation is updated.
> > > > > > # Signing and checksums must be cared in that case.
> > > > > >
> > > > > >
> > > > > > In addition, I got following warnings when I extracted the binary
> > > > > tarballs
> > > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz)
> on
> > > > > > Ubuntu.::
> > > > > >
> > > > > >
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > > tar: Ignoring unknown extended header keyword
> > > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > >
> > > > > >
> > > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > > It would be nice to fix too.
> > > > > >
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > > Thanks Masatake and Steve,
> > > > > > >
> > > > > > > Here's the PR for updating the top page of the site
> > > > > > https://github.com/apache/hadoop/pull/7887 <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > > > > > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887>> , if there's
> anything
> > > else
> > > > > we
> > > > > > think needs to go in there, let me know, otherwise I will merge
> by
> > > > > tomorrow.
> > > > > > >
> > > > > > > Since I have to regenerate the site and changelog, do I need to
> > do
> > > > the
> > > > > > whole build again or is there an easier way to do that? (just by
> > > > > > regenerating the site manually and updating the artifacts in SVN
> > > > maybe?)
> > > > > > >
> > > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz
> and
> > > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > > >
> > > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > > iwasakims@oss.nttdata.com
> > > > <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>
> > > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> > iwasakims@oss.nttdata.com
> > > >>>>
> > > > wrote:
> > > > > > >
> > > > > > >
> > > > > > > CAUTION: This email originated from outside of the
> organization.
> > Do
> > > > not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks for the update, Ahmar
> > > > > > >
> > > > > > >
> > > > > > > The binary artifacts basically looks good.
> > > > > > > I have some comments about inconsistencies.
> > > > > > >
> > > > > > >
> > > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > > Version/s". I updated them.
> > > > > > >
> > > > > > >
> > > > > > > Jira/Git commit message diff starting:
> > > > > > ##############################################
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using
> SSE-KMS
> > > > #7816
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A:
> Updates
> > > AAL
> > > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > > LocalDirAllocator
> > > > > > still doesn't always recover from directory deletion (#7683)
> > (#7685)
> > > > > > > ...
> > > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > > commons-cli
> > > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > > >
> > > > > > >
> > > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > > >
> > > > > >
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > > >
> > > > > > ;>
> > > > > > >
> > > > > > >
> > > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > > >
> > > > > > >
> > > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > > >
> > > > > > >
> > > > > > > Masatake Iwasaki
> > > > > > >
> > > > > > >
> > > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > > >> Hey all,
> > > > > > >>
> > > > > > >> Just a reminder about this. Would be great to get some votes
> in
> > > asap
> > > > > and
> > > > > > >> hopefully finalise 3.4.2.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Ahmar
> > > > > > >>
> > > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > > <ma...@apache.org>
> > > > > <mailto:
> > > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> > ahmar@apache.org
> > > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > > ahmar@apache.org>>>>
> > > > > > wrote:
> > > > > > >>
> > > > > > >>> Hey all,
> > > > > > >>>
> > > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > > >>>
> > > > > > >>> There are a couple of things to note:
> > > > > > >>>
> > > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and
> Wei-Chiu
> > > > > Chuang
> > > > > > for
> > > > > > >>> working on the fix.
> > > > > > >>>
> > > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > > bundle
> > > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is
> now
> > > > > ~600MB,
> > > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > > uploaded
> > > > > to
> > > > > > SVN.
> > > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > > >>>
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > <
> > > > > >
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > > >
> > > > > >
> > > > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > ;>
> > > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > > >
> > > > > > ;>,
> > > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> > be
> > > > > > updated
> > > > > > >>> with these instructions.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> The RC is available at:
> > > > > > >>>
> > > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>
> ;>
> > > > > > >>>
> > > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > > >>>
> > > > > > >>> The maven artifacts are staged at:
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > ;>
> > > > > <
> > > > > >
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > > >
> > > > > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> You can find my public key
> > > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > > at:
> > > > > > >>>
> > > > > > >>>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> ;>
> > <
> > > >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > > ;>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> This release has been created off of branch-3.4.
> > > > > > >>>
> > > > > > >>> Thanks,
> > > > > > >>> Ahmar
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > > For additional commands, e-mail:
> common-dev-help@hadoop.apache.org
> > > > <ma...@hadoop.apache.org>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > >
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Mukund Madhav Thakur <mt...@cloudera.com.INVALID>.
+1 binding
Verified signature and checksums.
Compiled the full source tree successfully.
Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
on us-west2 bucket which verified dropping in of aws sdk library.
Also verified access to abfs containers.
Compiled gcs connector using this release.
Thanks Ahmar for driving the release.
PS: I was expecting a new email for RC3 as RC2 had some problems. I think
the norm is to create a new email for every RC vote as per my
understanding.
On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> +1 binding
>
> * downloaded, checked signing and hashes of binary, arm binary and source
> artifacts
> * full clean build of source
> * aws test runs against s3 london and dell ecs. One oddity in building from
> the untarred source tree, Terasort failing with history server not
> starting. This did work in my git copy of the source so I'm assuming I've
> got the core-site/auth-keys wrong (the ones which get copied in). Still,
> the cluster tests run, which is an improvement over trunk right now
> * full s3a command line testing as per my qualifying and sdk doc.
>
>
> Steve
>
>
>
>
>
> On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> wrote:
>
> > +1 (non-binding)
> >
> > * Verified sha512 checksum was correct for source tarball
> > * Verified signature was correct for source tarball (not verified trust)
> > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > non-blocker for release as
> > https://issues.apache.org/jira/browse/HADOOP-19556
> > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > passing against Amazon S3 in eu-west-1
> >
> > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > tarihinde şunu yazdı:
> >
> > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > included in the release yet, if the RC fails the vote, will include it
> in
> > > the next one.
> > >
> > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > anujmodi2011@gmail.com>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi, Ahmar
> > >
> > >
> > > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> > on
> > > RC.
> > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > https://github.com/apache/hadoop/pull/7888>
> > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > https://github.com/apache/hadoop/pull/7889>
> > >
> > >
> > > Feel free to merge them as per the release timelines.
> > >
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> <stevel@cloudera.com.inva
> > > <ma...@cloudera.com.inva>lid>
> > > wrote:
> > >
> > >
> > > > just discovering as I do the third party compatibility testing of the
> > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > broken
> > > > -nothing to worry about though as its a new feature so not a
> regression
> > > >
> > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> <ahmarsu@amazon.co.uk.inva
> > > <ma...@amazon.co.uk.inva>lid>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > I've fixed the following issues:
> > > > > 1) Issues with changelog, release notes and landing page. Created a
> > new
> > > > > build so they now include all the JIRA's that were missing before.
> > > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > > unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > message when untarring
> > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > >
> > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > >
> > > > > The RC is available at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > >
> > > > > The git tag is release-3.4.2-RC3, commit
> > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > >
> > > > > The maven artifacts are staged at:
> > > > >
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > >
> > > > > You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > >
> > > > >
> > > > > This release has been created off of branch-3.4.
> > > > >
> > > > > Thanks,
> > > > > Ahmar
> > > > >
> > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>
> > > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > >
> > > > >
> > > > > I think just running the create-release[1] script could be an easy
> > way
> > > > > while it does not provide options for building only site or source
> > > > > tarballs (now).
> > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > RELEASENOTES,
> > > > > it would be nice to update the binary tarballs.
> > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> tarballs
> > > > > # could be ok if only documentation is updated.
> > > > > # Signing and checksums must be cared in that case.
> > > > >
> > > > >
> > > > > In addition, I got following warnings when I extracted the binary
> > > > tarballs
> > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > > Ubuntu.::
> > > > >
> > > > >
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > >
> > > > >
> > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > It would be nice to fix too.
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > >
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > Thanks Masatake and Steve,
> > > > > >
> > > > > > Here's the PR for updating the top page of the site
> > > > > https://github.com/apache/hadoop/pull/7887 <
> > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> > else
> > > > we
> > > > > think needs to go in there, let me know, otherwise I will merge by
> > > > tomorrow.
> > > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > > >
> > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > >
> > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>>
> > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks for the update, Ahmar
> > > > > >
> > > > > >
> > > > > > The binary artifacts basically looks good.
> > > > > > I have some comments about inconsistencies.
> > > > > >
> > > > > >
> > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > Version/s". I updated them.
> > > > > >
> > > > > >
> > > > > > Jira/Git commit message diff starting:
> > > > > ##############################################
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > > #7816
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> > AAL
> > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > LocalDirAllocator
> > > > > still doesn't always recover from directory deletion (#7683)
> (#7685)
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > commons-cli
> > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > >
> > > > > >
> > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > >
> > > > > ;>
> > > > > >
> > > > > >
> > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > >> Hey all,
> > > > > >>
> > > > > >> Just a reminder about this. Would be great to get some votes in
> > asap
> > > > and
> > > > > >> hopefully finalise 3.4.2.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Ahmar
> > > > > >>
> > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > <ma...@apache.org>
> > > > <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> ahmar@apache.org
> > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > ahmar@apache.org>>>>
> > > > > wrote:
> > > > > >>
> > > > > >>> Hey all,
> > > > > >>>
> > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > >>>
> > > > > >>> There are a couple of things to note:
> > > > > >>>
> > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > > Chuang
> > > > > for
> > > > > >>> working on the fix.
> > > > > >>>
> > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > bundle
> > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > > ~600MB,
> > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > uploaded
> > > > to
> > > > > SVN.
> > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > >>>
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > >
> > > > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > ;>
> > > > > <
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > >
> > > > > ;>,
> > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> be
> > > > > updated
> > > > > >>> with these instructions.
> > > > > >>>
> > > > > >>>
> > > > > >>> The RC is available at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > > >>>
> > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > >>>
> > > > > >>> The maven artifacts are staged at:
> > > > > >>>
> > > > > >>>
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > >
> > > > > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> <
> > >
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> This release has been created off of branch-3.4.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Ahmar
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Mukund Madhav Thakur <mt...@cloudera.com.INVALID>.
+1 binding
Verified signature and checksums.
Compiled the full source tree successfully.
Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
on us-west2 bucket which verified dropping in of aws sdk library.
Also verified access to abfs containers.
Compiled gcs connector using this release.
Thanks Ahmar for driving the release.
PS: I was expecting a new email for RC3 as RC2 had some problems. I think
the norm is to create a new email for every RC vote as per my
understanding.
On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> +1 binding
>
> * downloaded, checked signing and hashes of binary, arm binary and source
> artifacts
> * full clean build of source
> * aws test runs against s3 london and dell ecs. One oddity in building from
> the untarred source tree, Terasort failing with history server not
> starting. This did work in my git copy of the source so I'm assuming I've
> got the core-site/auth-keys wrong (the ones which get copied in). Still,
> the cluster tests run, which is an improvement over trunk right now
> * full s3a command line testing as per my qualifying and sdk doc.
>
>
> Steve
>
>
>
>
>
> On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> wrote:
>
> > +1 (non-binding)
> >
> > * Verified sha512 checksum was correct for source tarball
> > * Verified signature was correct for source tarball (not verified trust)
> > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > non-blocker for release as
> > https://issues.apache.org/jira/browse/HADOOP-19556
> > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > passing against Amazon S3 in eu-west-1
> >
> > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > tarihinde şunu yazdı:
> >
> > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > included in the release yet, if the RC fails the vote, will include it
> in
> > > the next one.
> > >
> > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > anujmodi2011@gmail.com>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi, Ahmar
> > >
> > >
> > > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> > on
> > > RC.
> > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > https://github.com/apache/hadoop/pull/7888>
> > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > https://github.com/apache/hadoop/pull/7889>
> > >
> > >
> > > Feel free to merge them as per the release timelines.
> > >
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> <stevel@cloudera.com.inva
> > > <ma...@cloudera.com.inva>lid>
> > > wrote:
> > >
> > >
> > > > just discovering as I do the third party compatibility testing of the
> > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > broken
> > > > -nothing to worry about though as its a new feature so not a
> regression
> > > >
> > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> <ahmarsu@amazon.co.uk.inva
> > > <ma...@amazon.co.uk.inva>lid>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > I've fixed the following issues:
> > > > > 1) Issues with changelog, release notes and landing page. Created a
> > new
> > > > > build so they now include all the JIRA's that were missing before.
> > > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > > unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > message when untarring
> > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > >
> > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > >
> > > > > The RC is available at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > >
> > > > > The git tag is release-3.4.2-RC3, commit
> > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > >
> > > > > The maven artifacts are staged at:
> > > > >
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > >
> > > > > You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > >
> > > > >
> > > > > This release has been created off of branch-3.4.
> > > > >
> > > > > Thanks,
> > > > > Ahmar
> > > > >
> > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>
> > > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > >
> > > > >
> > > > > I think just running the create-release[1] script could be an easy
> > way
> > > > > while it does not provide options for building only site or source
> > > > > tarballs (now).
> > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > RELEASENOTES,
> > > > > it would be nice to update the binary tarballs.
> > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> tarballs
> > > > > # could be ok if only documentation is updated.
> > > > > # Signing and checksums must be cared in that case.
> > > > >
> > > > >
> > > > > In addition, I got following warnings when I extracted the binary
> > > > tarballs
> > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > > Ubuntu.::
> > > > >
> > > > >
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > >
> > > > >
> > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > It would be nice to fix too.
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > >
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > Thanks Masatake and Steve,
> > > > > >
> > > > > > Here's the PR for updating the top page of the site
> > > > > https://github.com/apache/hadoop/pull/7887 <
> > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> > else
> > > > we
> > > > > think needs to go in there, let me know, otherwise I will merge by
> > > > tomorrow.
> > > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > > >
> > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > >
> > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>>
> > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks for the update, Ahmar
> > > > > >
> > > > > >
> > > > > > The binary artifacts basically looks good.
> > > > > > I have some comments about inconsistencies.
> > > > > >
> > > > > >
> > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > Version/s". I updated them.
> > > > > >
> > > > > >
> > > > > > Jira/Git commit message diff starting:
> > > > > ##############################################
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > > #7816
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> > AAL
> > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > LocalDirAllocator
> > > > > still doesn't always recover from directory deletion (#7683)
> (#7685)
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > commons-cli
> > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > >
> > > > > >
> > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > >
> > > > > ;>
> > > > > >
> > > > > >
> > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > >> Hey all,
> > > > > >>
> > > > > >> Just a reminder about this. Would be great to get some votes in
> > asap
> > > > and
> > > > > >> hopefully finalise 3.4.2.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Ahmar
> > > > > >>
> > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > <ma...@apache.org>
> > > > <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> ahmar@apache.org
> > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > ahmar@apache.org>>>>
> > > > > wrote:
> > > > > >>
> > > > > >>> Hey all,
> > > > > >>>
> > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > >>>
> > > > > >>> There are a couple of things to note:
> > > > > >>>
> > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > > Chuang
> > > > > for
> > > > > >>> working on the fix.
> > > > > >>>
> > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > bundle
> > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > > ~600MB,
> > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > uploaded
> > > > to
> > > > > SVN.
> > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > >>>
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > >
> > > > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > ;>
> > > > > <
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > >
> > > > > ;>,
> > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> be
> > > > > updated
> > > > > >>> with these instructions.
> > > > > >>>
> > > > > >>>
> > > > > >>> The RC is available at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > > >>>
> > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > >>>
> > > > > >>> The maven artifacts are staged at:
> > > > > >>>
> > > > > >>>
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > >
> > > > > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> <
> > >
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> This release has been created off of branch-3.4.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Ahmar
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Mukund Madhav Thakur <mt...@cloudera.com.INVALID>.
+1 binding
Verified signature and checksums.
Compiled the full source tree successfully.
Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
on us-west2 bucket which verified dropping in of aws sdk library.
Also verified access to abfs containers.
Compiled gcs connector using this release.
Thanks Ahmar for driving the release.
PS: I was expecting a new email for RC3 as RC2 had some problems. I think
the norm is to create a new email for every RC vote as per my
understanding.
On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> +1 binding
>
> * downloaded, checked signing and hashes of binary, arm binary and source
> artifacts
> * full clean build of source
> * aws test runs against s3 london and dell ecs. One oddity in building from
> the untarred source tree, Terasort failing with history server not
> starting. This did work in my git copy of the source so I'm assuming I've
> got the core-site/auth-keys wrong (the ones which get copied in). Still,
> the cluster tests run, which is an improvement over trunk right now
> * full s3a command line testing as per my qualifying and sdk doc.
>
>
> Steve
>
>
>
>
>
> On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> wrote:
>
> > +1 (non-binding)
> >
> > * Verified sha512 checksum was correct for source tarball
> > * Verified signature was correct for source tarball (not verified trust)
> > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > non-blocker for release as
> > https://issues.apache.org/jira/browse/HADOOP-19556
> > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > passing against Amazon S3 in eu-west-1
> >
> > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > tarihinde şunu yazdı:
> >
> > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > included in the release yet, if the RC fails the vote, will include it
> in
> > > the next one.
> > >
> > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > anujmodi2011@gmail.com>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi, Ahmar
> > >
> > >
> > > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> > on
> > > RC.
> > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > https://github.com/apache/hadoop/pull/7888>
> > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > https://github.com/apache/hadoop/pull/7889>
> > >
> > >
> > > Feel free to merge them as per the release timelines.
> > >
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> <stevel@cloudera.com.inva
> > > <ma...@cloudera.com.inva>lid>
> > > wrote:
> > >
> > >
> > > > just discovering as I do the third party compatibility testing of the
> > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > broken
> > > > -nothing to worry about though as its a new feature so not a
> regression
> > > >
> > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> <ahmarsu@amazon.co.uk.inva
> > > <ma...@amazon.co.uk.inva>lid>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > I've fixed the following issues:
> > > > > 1) Issues with changelog, release notes and landing page. Created a
> > new
> > > > > build so they now include all the JIRA's that were missing before.
> > > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > > unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > message when untarring
> > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > >
> > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > >
> > > > > The RC is available at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > >
> > > > > The git tag is release-3.4.2-RC3, commit
> > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > >
> > > > > The maven artifacts are staged at:
> > > > >
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > >
> > > > > You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > >
> > > > >
> > > > > This release has been created off of branch-3.4.
> > > > >
> > > > > Thanks,
> > > > > Ahmar
> > > > >
> > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>
> > > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > >
> > > > >
> > > > > I think just running the create-release[1] script could be an easy
> > way
> > > > > while it does not provide options for building only site or source
> > > > > tarballs (now).
> > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > RELEASENOTES,
> > > > > it would be nice to update the binary tarballs.
> > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> tarballs
> > > > > # could be ok if only documentation is updated.
> > > > > # Signing and checksums must be cared in that case.
> > > > >
> > > > >
> > > > > In addition, I got following warnings when I extracted the binary
> > > > tarballs
> > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > > Ubuntu.::
> > > > >
> > > > >
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > >
> > > > >
> > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > It would be nice to fix too.
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > >
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > Thanks Masatake and Steve,
> > > > > >
> > > > > > Here's the PR for updating the top page of the site
> > > > > https://github.com/apache/hadoop/pull/7887 <
> > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> > else
> > > > we
> > > > > think needs to go in there, let me know, otherwise I will merge by
> > > > tomorrow.
> > > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > > >
> > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > >
> > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>>
> > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks for the update, Ahmar
> > > > > >
> > > > > >
> > > > > > The binary artifacts basically looks good.
> > > > > > I have some comments about inconsistencies.
> > > > > >
> > > > > >
> > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > Version/s". I updated them.
> > > > > >
> > > > > >
> > > > > > Jira/Git commit message diff starting:
> > > > > ##############################################
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > > #7816
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> > AAL
> > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > LocalDirAllocator
> > > > > still doesn't always recover from directory deletion (#7683)
> (#7685)
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > commons-cli
> > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > >
> > > > > >
> > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > >
> > > > > ;>
> > > > > >
> > > > > >
> > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > >> Hey all,
> > > > > >>
> > > > > >> Just a reminder about this. Would be great to get some votes in
> > asap
> > > > and
> > > > > >> hopefully finalise 3.4.2.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Ahmar
> > > > > >>
> > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > <ma...@apache.org>
> > > > <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> ahmar@apache.org
> > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > ahmar@apache.org>>>>
> > > > > wrote:
> > > > > >>
> > > > > >>> Hey all,
> > > > > >>>
> > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > >>>
> > > > > >>> There are a couple of things to note:
> > > > > >>>
> > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > > Chuang
> > > > > for
> > > > > >>> working on the fix.
> > > > > >>>
> > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > bundle
> > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > > ~600MB,
> > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > uploaded
> > > > to
> > > > > SVN.
> > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > >>>
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > >
> > > > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > ;>
> > > > > <
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > >
> > > > > ;>,
> > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> be
> > > > > updated
> > > > > >>> with these instructions.
> > > > > >>>
> > > > > >>>
> > > > > >>> The RC is available at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > > >>>
> > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > >>>
> > > > > >>> The maven artifacts are staged at:
> > > > > >>>
> > > > > >>>
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > >
> > > > > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> <
> > >
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> This release has been created off of branch-3.4.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Ahmar
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Mukund Madhav Thakur <mt...@cloudera.com.INVALID>.
+1 binding
Verified signature and checksums.
Compiled the full source tree successfully.
Downloaded and untar the arm binary on Mac OS M3 and ran hadoop fs command
on us-west2 bucket which verified dropping in of aws sdk library.
Also verified access to abfs containers.
Compiled gcs connector using this release.
Thanks Ahmar for driving the release.
PS: I was expecting a new email for RC3 as RC2 had some problems. I think
the norm is to create a new email for every RC vote as per my
understanding.
On Mon, Aug 25, 2025 at 4:31 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> +1 binding
>
> * downloaded, checked signing and hashes of binary, arm binary and source
> artifacts
> * full clean build of source
> * aws test runs against s3 london and dell ecs. One oddity in building from
> the untarred source tree, Terasort failing with history server not
> starting. This did work in my git copy of the source so I'm assuming I've
> got the core-site/auth-keys wrong (the ones which get copied in). Still,
> the cluster tests run, which is an improvement over trunk right now
> * full s3a command line testing as per my qualifying and sdk doc.
>
>
> Steve
>
>
>
>
>
> On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
> wrote:
>
> > +1 (non-binding)
> >
> > * Verified sha512 checksum was correct for source tarball
> > * Verified signature was correct for source tarball (not verified trust)
> > * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> > * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> > ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> > non-blocker for release as
> > https://issues.apache.org/jira/browse/HADOOP-19556
> > * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> > passing against Amazon S3 in eu-west-1
> >
> > Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> > tarihinde şunu yazdı:
> >
> > > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > > included in the release yet, if the RC fails the vote, will include it
> in
> > > the next one.
> > >
> > > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > > anujmodi2011@gmail.com>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Hi, Ahmar
> > >
> > >
> > > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> > on
> > > RC.
> > > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > > https://github.com/apache/hadoop/pull/7888>
> > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > > https://github.com/apache/hadoop/pull/7889>
> > >
> > >
> > > Feel free to merge them as per the release timelines.
> > >
> > >
> > > Thanks
> > >
> > >
> > > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran
> <stevel@cloudera.com.inva
> > > <ma...@cloudera.com.inva>lid>
> > > wrote:
> > >
> > >
> > > > just discovering as I do the third party compatibility testing of the
> > > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > > broken
> > > > -nothing to worry about though as its a new feature so not a
> regression
> > > >
> > > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar
> <ahmarsu@amazon.co.uk.inva
> > > <ma...@amazon.co.uk.inva>lid>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > I've fixed the following issues:
> > > > > 1) Issues with changelog, release notes and landing page. Created a
> > new
> > > > > build so they now include all the JIRA's that were missing before.
> > > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > > unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > message when untarring
> > > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > > >
> > > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > > >
> > > > > The RC is available at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > > >
> > > > > The git tag is release-3.4.2-RC3, commit
> > > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > > >
> > > > > The maven artifacts are staged at:
> > > > >
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > > >
> > > > > You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > > >
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > > >
> > > > >
> > > > > This release has been created off of branch-3.4.
> > > > >
> > > > > Thanks,
> > > > > Ahmar
> > > > >
> > > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>
> > > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > >
> > > > >
> > > > > I think just running the create-release[1] script could be an easy
> > way
> > > > > while it does not provide options for building only site or source
> > > > > tarballs (now).
> > > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > > RELEASENOTES,
> > > > > it would be nice to update the binary tarballs.
> > > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary
> tarballs
> > > > > # could be ok if only documentation is updated.
> > > > > # Signing and checksums must be cared in that case.
> > > > >
> > > > >
> > > > > In addition, I got following warnings when I extracted the binary
> > > > tarballs
> > > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > > Ubuntu.::
> > > > >
> > > > >
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > > tar: Ignoring unknown extended header keyword
> > > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > >
> > > > >
> > > > > I guess the cause is modifying the tar files on OSX environment.
> > > > > It would be nice to fix too.
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > > >
> > > > > >
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > > Thanks Masatake and Steve,
> > > > > >
> > > > > > Here's the PR for updating the top page of the site
> > > > > https://github.com/apache/hadoop/pull/7887 <
> > > https://github.com/apache/hadoop/pull/7887> <
> > > > > https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> > else
> > > > we
> > > > > think needs to go in there, let me know, otherwise I will merge by
> > > > tomorrow.
> > > > > >
> > > > > > Since I have to regenerate the site and changelog, do I need to
> do
> > > the
> > > > > whole build again or is there an easier way to do that? (just by
> > > > > regenerating the site manually and updating the artifacts in SVN
> > > maybe?)
> > > > > >
> > > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > > >
> > > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> > iwasakims@oss.nttdata.com
> > > <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > > <mailto:iwasakims@oss.nttdata.com <mailto:
> iwasakims@oss.nttdata.com
> > >>>>
> > > wrote:
> > > > > >
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> Do
> > > not
> > > > > click links or open attachments unless you can confirm the sender
> and
> > > > know
> > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks for the update, Ahmar
> > > > > >
> > > > > >
> > > > > > The binary artifacts basically looks good.
> > > > > > I have some comments about inconsistencies.
> > > > > >
> > > > > >
> > > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > > Version/s". I updated them.
> > > > > >
> > > > > >
> > > > > > Jira/Git commit message diff starting:
> > > > > ##############################################
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > > #7816
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> > AAL
> > > > > version to 1.2.1 (#7807) (#7815)
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > > LocalDirAllocator
> > > > > still doesn't always recover from directory deletion (#7683)
> (#7685)
> > > > > > ...
> > > > > > Jira not present with version: 3.4.2. Commit:
> > > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > > commons-cli
> > > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > > >
> > > > > >
> > > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > > >
> > > > >
> > > > > <
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > > >
> > > > > ;>
> > > > > >
> > > > > >
> > > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > > >
> > > > > >
> > > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > > >
> > > > > >
> > > > > > Masatake Iwasaki
> > > > > >
> > > > > >
> > > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > > >> Hey all,
> > > > > >>
> > > > > >> Just a reminder about this. Would be great to get some votes in
> > asap
> > > > and
> > > > > >> hopefully finalise 3.4.2.
> > > > > >>
> > > > > >> Thanks,
> > > > > >> Ahmar
> > > > > >>
> > > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > > <ma...@apache.org>
> > > > <mailto:
> > > > > ahmar@apache.org <ma...@apache.org>> <mailto:
> ahmar@apache.org
> > > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > > ahmar@apache.org>>>>
> > > > > wrote:
> > > > > >>
> > > > > >>> Hey all,
> > > > > >>>
> > > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > > >>>
> > > > > >>> There are a couple of things to note:
> > > > > >>>
> > > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > > Chuang
> > > > > for
> > > > > >>> working on the fix.
> > > > > >>>
> > > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > > bundle
> > > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > > ~600MB,
> > > > > >>> which makes the size of tar > 1GB, and it can no longer be
> > uploaded
> > > > to
> > > > > SVN.
> > > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > > >>>
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > <
> > > > >
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> > >
> > > > >
> > > > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > ;>
> > > > > <
> > > > >
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > > >
> > > > > ;>,
> > > > > >>> and drop it into /share/hadoop/common/lib. Release notes will
> be
> > > > > updated
> > > > > >>> with these instructions.
> > > > > >>>
> > > > > >>>
> > > > > >>> The RC is available at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > > >>>
> > > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > > >>>
> > > > > >>> The maven artifacts are staged at:
> > > > > >>>
> > > > > >>>
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > ;>
> > > > <
> > > > >
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > > >
> > > > > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> You can find my public key
> > > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > > at:
> > > > > >>>
> > > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> <
> > >
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> > ;>
> > > > > >>>
> > > > > >>>
> > > > > >>> This release has been created off of branch-3.4.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Ahmar
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > > <ma...@hadoop.apache.org>
> > > > >
> > > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
+1 binding
* downloaded, checked signing and hashes of binary, arm binary and source
artifacts
* full clean build of source
* aws test runs against s3 london and dell ecs. One oddity in building from
the untarred source tree, Terasort failing with history server not
starting. This did work in my git copy of the source so I'm assuming I've
got the core-site/auth-keys wrong (the ones which get copied in). Still,
the cluster tests run, which is an improvement over trunk right now
* full s3a command line testing as per my qualifying and sdk doc.
Steve
On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
wrote:
> +1 (non-binding)
>
> * Verified sha512 checksum was correct for source tarball
> * Verified signature was correct for source tarball (not verified trust)
> * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> non-blocker for release as
> https://issues.apache.org/jira/browse/HADOOP-19556
> * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> passing against Amazon S3 in eu-west-1
>
> Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> tarihinde şunu yazdı:
>
> > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > included in the release yet, if the RC fails the vote, will include it in
> > the next one.
> >
> > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > anujmodi2011@gmail.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > Hi, Ahmar
> >
> >
> > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> on
> > RC.
> > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > https://github.com/apache/hadoop/pull/7888>
> > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > https://github.com/apache/hadoop/pull/7889>
> >
> >
> > Feel free to merge them as per the release timelines.
> >
> >
> > Thanks
> >
> >
> > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> > <ma...@cloudera.com.inva>lid>
> > wrote:
> >
> >
> > > just discovering as I do the third party compatibility testing of the
> > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > broken
> > > -nothing to worry about though as its a new feature so not a regression
> > >
> > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> > <ma...@amazon.co.uk.inva>lid>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > I've fixed the following issues:
> > > > 1) Issues with changelog, release notes and landing page. Created a
> new
> > > > build so they now include all the JIRA's that were missing before.
> > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > message when untarring
> > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > >
> > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > >
> > > > The RC is available at:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > >
> > > > The git tag is release-3.4.2-RC3, commit
> > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > >
> > > > The maven artifacts are staged at:
> > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > >
> > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > > >
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > >
> > > >
> > > > This release has been created off of branch-3.4.
> > > >
> > > > Thanks,
> > > > Ahmar
> > > >
> > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>
> > wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > >
> > > >
> > > > I think just running the create-release[1] script could be an easy
> way
> > > > while it does not provide options for building only site or source
> > > > tarballs (now).
> > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > RELEASENOTES,
> > > > it would be nice to update the binary tarballs.
> > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > > # could be ok if only documentation is updated.
> > > > # Signing and checksums must be cared in that case.
> > > >
> > > >
> > > > In addition, I got following warnings when I extracted the binary
> > > tarballs
> > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > Ubuntu.::
> > > >
> > > >
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > >
> > > >
> > > > I guess the cause is modifying the tar files on OSX environment.
> > > > It would be nice to fix too.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > >
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > Thanks Masatake and Steve,
> > > > >
> > > > > Here's the PR for updating the top page of the site
> > > > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> else
> > > we
> > > > think needs to go in there, let me know, otherwise I will merge by
> > > tomorrow.
> > > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > > >
> > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > >
> > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>>
> > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks for the update, Ahmar
> > > > >
> > > > >
> > > > > The binary artifacts basically looks good.
> > > > > I have some comments about inconsistencies.
> > > > >
> > > > >
> > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > Version/s". I updated them.
> > > > >
> > > > >
> > > > > Jira/Git commit message diff starting:
> > > > ##############################################
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > #7816
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> AAL
> > > > version to 1.2.1 (#7807) (#7815)
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > LocalDirAllocator
> > > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > commons-cli
> > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > >
> > > > >
> > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > >
> > > > ;>
> > > > >
> > > > >
> > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > >> Hey all,
> > > > >>
> > > > >> Just a reminder about this. Would be great to get some votes in
> asap
> > > and
> > > > >> hopefully finalise 3.4.2.
> > > > >>
> > > > >> Thanks,
> > > > >> Ahmar
> > > > >>
> > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > <ma...@apache.org>
> > > <mailto:
> > > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > ahmar@apache.org>>>>
> > > > wrote:
> > > > >>
> > > > >>> Hey all,
> > > > >>>
> > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > >>>
> > > > >>> There are a couple of things to note:
> > > > >>>
> > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > Chuang
> > > > for
> > > > >>> working on the fix.
> > > > >>>
> > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > bundle
> > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > ~600MB,
> > > > >>> which makes the size of tar > 1GB, and it can no longer be
> uploaded
> > > to
> > > > SVN.
> > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > >>>
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > >
> > > > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > >
> > > > ;>,
> > > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > > updated
> > > > >>> with these instructions.
> > > > >>>
> > > > >>>
> > > > >>> The RC is available at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > >>>
> > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > >>>
> > > > >>> The maven artifacts are staged at:
> > > > >>>
> > > > >>>
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > >
> > > > ;>
> > > > >>>
> > > > >>>
> > > > >>> You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> ;>
> > > > >>>
> > > > >>>
> > > > >>> This release has been created off of branch-3.4.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Ahmar
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > >
> > >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
+1 binding
* downloaded, checked signing and hashes of binary, arm binary and source
artifacts
* full clean build of source
* aws test runs against s3 london and dell ecs. One oddity in building from
the untarred source tree, Terasort failing with history server not
starting. This did work in my git copy of the source so I'm assuming I've
got the core-site/auth-keys wrong (the ones which get copied in). Still,
the cluster tests run, which is an improvement over trunk right now
* full s3a command line testing as per my qualifying and sdk doc.
Steve
On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
wrote:
> +1 (non-binding)
>
> * Verified sha512 checksum was correct for source tarball
> * Verified signature was correct for source tarball (not verified trust)
> * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> non-blocker for release as
> https://issues.apache.org/jira/browse/HADOOP-19556
> * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> passing against Amazon S3 in eu-west-1
>
> Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> tarihinde şunu yazdı:
>
> > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > included in the release yet, if the RC fails the vote, will include it in
> > the next one.
> >
> > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > anujmodi2011@gmail.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > Hi, Ahmar
> >
> >
> > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> on
> > RC.
> > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > https://github.com/apache/hadoop/pull/7888>
> > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > https://github.com/apache/hadoop/pull/7889>
> >
> >
> > Feel free to merge them as per the release timelines.
> >
> >
> > Thanks
> >
> >
> > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> > <ma...@cloudera.com.inva>lid>
> > wrote:
> >
> >
> > > just discovering as I do the third party compatibility testing of the
> > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > broken
> > > -nothing to worry about though as its a new feature so not a regression
> > >
> > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> > <ma...@amazon.co.uk.inva>lid>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > I've fixed the following issues:
> > > > 1) Issues with changelog, release notes and landing page. Created a
> new
> > > > build so they now include all the JIRA's that were missing before.
> > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > message when untarring
> > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > >
> > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > >
> > > > The RC is available at:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > >
> > > > The git tag is release-3.4.2-RC3, commit
> > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > >
> > > > The maven artifacts are staged at:
> > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > >
> > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > > >
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > >
> > > >
> > > > This release has been created off of branch-3.4.
> > > >
> > > > Thanks,
> > > > Ahmar
> > > >
> > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>
> > wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > >
> > > >
> > > > I think just running the create-release[1] script could be an easy
> way
> > > > while it does not provide options for building only site or source
> > > > tarballs (now).
> > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > RELEASENOTES,
> > > > it would be nice to update the binary tarballs.
> > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > > # could be ok if only documentation is updated.
> > > > # Signing and checksums must be cared in that case.
> > > >
> > > >
> > > > In addition, I got following warnings when I extracted the binary
> > > tarballs
> > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > Ubuntu.::
> > > >
> > > >
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > >
> > > >
> > > > I guess the cause is modifying the tar files on OSX environment.
> > > > It would be nice to fix too.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > >
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > Thanks Masatake and Steve,
> > > > >
> > > > > Here's the PR for updating the top page of the site
> > > > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> else
> > > we
> > > > think needs to go in there, let me know, otherwise I will merge by
> > > tomorrow.
> > > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > > >
> > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > >
> > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>>
> > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks for the update, Ahmar
> > > > >
> > > > >
> > > > > The binary artifacts basically looks good.
> > > > > I have some comments about inconsistencies.
> > > > >
> > > > >
> > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > Version/s". I updated them.
> > > > >
> > > > >
> > > > > Jira/Git commit message diff starting:
> > > > ##############################################
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > #7816
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> AAL
> > > > version to 1.2.1 (#7807) (#7815)
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > LocalDirAllocator
> > > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > commons-cli
> > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > >
> > > > >
> > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > >
> > > > ;>
> > > > >
> > > > >
> > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > >> Hey all,
> > > > >>
> > > > >> Just a reminder about this. Would be great to get some votes in
> asap
> > > and
> > > > >> hopefully finalise 3.4.2.
> > > > >>
> > > > >> Thanks,
> > > > >> Ahmar
> > > > >>
> > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > <ma...@apache.org>
> > > <mailto:
> > > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > ahmar@apache.org>>>>
> > > > wrote:
> > > > >>
> > > > >>> Hey all,
> > > > >>>
> > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > >>>
> > > > >>> There are a couple of things to note:
> > > > >>>
> > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > Chuang
> > > > for
> > > > >>> working on the fix.
> > > > >>>
> > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > bundle
> > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > ~600MB,
> > > > >>> which makes the size of tar > 1GB, and it can no longer be
> uploaded
> > > to
> > > > SVN.
> > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > >>>
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > >
> > > > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > >
> > > > ;>,
> > > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > > updated
> > > > >>> with these instructions.
> > > > >>>
> > > > >>>
> > > > >>> The RC is available at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > >>>
> > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > >>>
> > > > >>> The maven artifacts are staged at:
> > > > >>>
> > > > >>>
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > >
> > > > ;>
> > > > >>>
> > > > >>>
> > > > >>> You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> ;>
> > > > >>>
> > > > >>>
> > > > >>> This release has been created off of branch-3.4.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Ahmar
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > >
> > >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
+1 binding
* downloaded, checked signing and hashes of binary, arm binary and source
artifacts
* full clean build of source
* aws test runs against s3 london and dell ecs. One oddity in building from
the untarred source tree, Terasort failing with history server not
starting. This did work in my git copy of the source so I'm assuming I've
got the core-site/auth-keys wrong (the ones which get copied in). Still,
the cluster tests run, which is an improvement over trunk right now
* full s3a command line testing as per my qualifying and sdk doc.
Steve
On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
wrote:
> +1 (non-binding)
>
> * Verified sha512 checksum was correct for source tarball
> * Verified signature was correct for source tarball (not verified trust)
> * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> non-blocker for release as
> https://issues.apache.org/jira/browse/HADOOP-19556
> * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> passing against Amazon S3 in eu-west-1
>
> Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> tarihinde şunu yazdı:
>
> > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > included in the release yet, if the RC fails the vote, will include it in
> > the next one.
> >
> > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > anujmodi2011@gmail.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > Hi, Ahmar
> >
> >
> > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> on
> > RC.
> > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > https://github.com/apache/hadoop/pull/7888>
> > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > https://github.com/apache/hadoop/pull/7889>
> >
> >
> > Feel free to merge them as per the release timelines.
> >
> >
> > Thanks
> >
> >
> > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> > <ma...@cloudera.com.inva>lid>
> > wrote:
> >
> >
> > > just discovering as I do the third party compatibility testing of the
> > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > broken
> > > -nothing to worry about though as its a new feature so not a regression
> > >
> > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> > <ma...@amazon.co.uk.inva>lid>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > I've fixed the following issues:
> > > > 1) Issues with changelog, release notes and landing page. Created a
> new
> > > > build so they now include all the JIRA's that were missing before.
> > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > message when untarring
> > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > >
> > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > >
> > > > The RC is available at:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > >
> > > > The git tag is release-3.4.2-RC3, commit
> > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > >
> > > > The maven artifacts are staged at:
> > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > >
> > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > > >
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > >
> > > >
> > > > This release has been created off of branch-3.4.
> > > >
> > > > Thanks,
> > > > Ahmar
> > > >
> > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>
> > wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > >
> > > >
> > > > I think just running the create-release[1] script could be an easy
> way
> > > > while it does not provide options for building only site or source
> > > > tarballs (now).
> > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > RELEASENOTES,
> > > > it would be nice to update the binary tarballs.
> > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > > # could be ok if only documentation is updated.
> > > > # Signing and checksums must be cared in that case.
> > > >
> > > >
> > > > In addition, I got following warnings when I extracted the binary
> > > tarballs
> > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > Ubuntu.::
> > > >
> > > >
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > >
> > > >
> > > > I guess the cause is modifying the tar files on OSX environment.
> > > > It would be nice to fix too.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > >
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > Thanks Masatake and Steve,
> > > > >
> > > > > Here's the PR for updating the top page of the site
> > > > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> else
> > > we
> > > > think needs to go in there, let me know, otherwise I will merge by
> > > tomorrow.
> > > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > > >
> > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > >
> > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>>
> > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks for the update, Ahmar
> > > > >
> > > > >
> > > > > The binary artifacts basically looks good.
> > > > > I have some comments about inconsistencies.
> > > > >
> > > > >
> > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > Version/s". I updated them.
> > > > >
> > > > >
> > > > > Jira/Git commit message diff starting:
> > > > ##############################################
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > #7816
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> AAL
> > > > version to 1.2.1 (#7807) (#7815)
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > LocalDirAllocator
> > > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > commons-cli
> > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > >
> > > > >
> > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > >
> > > > ;>
> > > > >
> > > > >
> > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > >> Hey all,
> > > > >>
> > > > >> Just a reminder about this. Would be great to get some votes in
> asap
> > > and
> > > > >> hopefully finalise 3.4.2.
> > > > >>
> > > > >> Thanks,
> > > > >> Ahmar
> > > > >>
> > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > <ma...@apache.org>
> > > <mailto:
> > > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > ahmar@apache.org>>>>
> > > > wrote:
> > > > >>
> > > > >>> Hey all,
> > > > >>>
> > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > >>>
> > > > >>> There are a couple of things to note:
> > > > >>>
> > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > Chuang
> > > > for
> > > > >>> working on the fix.
> > > > >>>
> > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > bundle
> > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > ~600MB,
> > > > >>> which makes the size of tar > 1GB, and it can no longer be
> uploaded
> > > to
> > > > SVN.
> > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > >>>
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > >
> > > > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > >
> > > > ;>,
> > > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > > updated
> > > > >>> with these instructions.
> > > > >>>
> > > > >>>
> > > > >>> The RC is available at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > >>>
> > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > >>>
> > > > >>> The maven artifacts are staged at:
> > > > >>>
> > > > >>>
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > >
> > > > ;>
> > > > >>>
> > > > >>>
> > > > >>> You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> ;>
> > > > >>>
> > > > >>>
> > > > >>> This release has been created off of branch-3.4.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Ahmar
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > >
> > >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
+1 binding
* downloaded, checked signing and hashes of binary, arm binary and source
artifacts
* full clean build of source
* aws test runs against s3 london and dell ecs. One oddity in building from
the untarred source tree, Terasort failing with history server not
starting. This did work in my git copy of the source so I'm assuming I've
got the core-site/auth-keys wrong (the ones which get copied in). Still,
the cluster tests run, which is an improvement over trunk right now
* full s3a command line testing as per my qualifying and sdk doc.
Steve
On Thu, 21 Aug 2025 at 15:20, Erdoğan Özkoca <er...@gmail.com>
wrote:
> +1 (non-binding)
>
> * Verified sha512 checksum was correct for source tarball
> * Verified signature was correct for source tarball (not verified trust)
> * Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
> * Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
> ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
> non-blocker for release as
> https://issues.apache.org/jira/browse/HADOOP-19556
> * Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
> passing against Amazon S3 in eu-west-1
>
> Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
> tarihinde şunu yazdı:
>
> > Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> > included in the release yet, if the RC fails the vote, will include it in
> > the next one.
> >
> > On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> > anujmodi2011@gmail.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > Hi, Ahmar
> >
> >
> > Here are the PRs for the fix raised for ABFS close() causing NPE raised
> on
> > RC.
> > PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> > https://github.com/apache/hadoop/pull/7888>
> > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> > https://github.com/apache/hadoop/pull/7889>
> >
> >
> > Feel free to merge them as per the release timelines.
> >
> >
> > Thanks
> >
> >
> > On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> > <ma...@cloudera.com.inva>lid>
> > wrote:
> >
> >
> > > just discovering as I do the third party compatibility testing of the
> > > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> > broken
> > > -nothing to worry about though as its a new feature so not a regression
> > >
> > > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> > <ma...@amazon.co.uk.inva>lid>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > I've fixed the following issues:
> > > > 1) Issues with changelog, release notes and landing page. Created a
> new
> > > > build so they now include all the JIRA's that were missing before.
> > > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > > unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > message when untarring
> > > > 3) Previous issues with sha512 and tar not being gzipped.
> > > >
> > > > RC3 for Hadoop 3.4.2 is now available for voting.
> > > >
> > > > The RC is available at:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > > >
> > > > The git tag is release-3.4.2-RC3, commit
> > > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > > >
> > > > The maven artifacts are staged at:
> > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > > >
> > > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > > >
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > > >
> > > >
> > > > This release has been created off of branch-3.4.
> > > >
> > > > Thanks,
> > > > Ahmar
> > > >
> > > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>
> > wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > >
> > > >
> > > > I think just running the create-release[1] script could be an easy
> way
> > > > while it does not provide options for building only site or source
> > > > tarballs (now).
> > > > Since the binary tarballs are containing affected CHANGELOG and
> > > > RELEASENOTES,
> > > > it would be nice to update the binary tarballs.
> > > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > > # could be ok if only documentation is updated.
> > > > # Signing and checksums must be cared in that case.
> > > >
> > > >
> > > > In addition, I got following warnings when I extracted the binary
> > > tarballs
> > > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > > Ubuntu.::
> > > >
> > > >
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > > tar: Ignoring unknown extended header keyword
> > > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > >
> > > >
> > > > I guess the cause is modifying the tar files on OSX environment.
> > > > It would be nice to fix too.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> > > > >
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > > Thanks Masatake and Steve,
> > > > >
> > > > > Here's the PR for updating the top page of the site
> > > > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> <
> > > > https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887>> , if there's anything
> else
> > > we
> > > > think needs to go in there, let me know, otherwise I will merge by
> > > tomorrow.
> > > > >
> > > > > Since I have to regenerate the site and changelog, do I need to do
> > the
> > > > whole build again or is there an easier way to do that? (just by
> > > > regenerating the site manually and updating the artifacts in SVN
> > maybe?)
> > > > >
> > > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > > >
> > > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <
> iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > > <mailto:iwasakims@oss.nttdata.com <mailto:iwasakims@oss.nttdata.com
> >>>>
> > wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > click links or open attachments unless you can confirm the sender and
> > > know
> > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks for the update, Ahmar
> > > > >
> > > > >
> > > > > The binary artifacts basically looks good.
> > > > > I have some comments about inconsistencies.
> > > > >
> > > > >
> > > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > > Version/s". I updated them.
> > > > >
> > > > >
> > > > > Jira/Git commit message diff starting:
> > > > ##############################################
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> > #7816
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates
> AAL
> > > > version to 1.2.1 (#7807) (#7815)
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> > LocalDirAllocator
> > > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > > ...
> > > > > Jira not present with version: 3.4.2. Commit:
> > > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > > commons-cli
> > > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > > >
> > > > >
> > > > > - The top page of site documentation was not updated for 3.4.2.
> > > > >
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > >
> > > >
> > > > <
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> > >
> > > > ;>
> > > > >
> > > > >
> > > > > - Some release artifact are inconsistent as Steve pointed out.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > > >
> > > > >
> > > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > > >
> > > > >
> > > > > Masatake Iwasaki
> > > > >
> > > > >
> > > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > > >> Hey all,
> > > > >>
> > > > >> Just a reminder about this. Would be great to get some votes in
> asap
> > > and
> > > > >> hopefully finalise 3.4.2.
> > > > >>
> > > > >> Thanks,
> > > > >> Ahmar
> > > > >>
> > > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> > <ma...@apache.org>
> > > <mailto:
> > > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> > <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> > ahmar@apache.org>>>>
> > > > wrote:
> > > > >>
> > > > >>> Hey all,
> > > > >>>
> > > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > > >>>
> > > > >>> There are a couple of things to note:
> > > > >>>
> > > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > > Chuang
> > > > for
> > > > >>> working on the fix.
> > > > >>>
> > > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> > bundle
> > > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > > ~600MB,
> > > > >>> which makes the size of tar > 1GB, and it can no longer be
> uploaded
> > > to
> > > > SVN.
> > > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > > >>>
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > <
> > > >
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > > >
> > > > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>
> > > > <
> > > >
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> > >
> > > > ;>,
> > > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > > updated
> > > > >>> with these instructions.
> > > > >>>
> > > > >>>
> > > > >>> The RC is available at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > > >>>
> > > > >>> The git tag is release-3.4.2-RC2, commit
> > > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > > >>>
> > > > >>> The maven artifacts are staged at:
> > > > >>>
> > > > >>>
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > > <
> > > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> > >
> > > > ;>
> > > > >>>
> > > > >>>
> > > > >>> You can find my public key
> > (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > > at:
> > > > >>>
> > > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>
> ;>
> > > > >>>
> > > > >>>
> > > > >>> This release has been created off of branch-3.4.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Ahmar
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > <ma...@hadoop.apache.org>
> > > >
> > >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Erdoğan Özkoca <er...@gmail.com>.
+1 (non-binding)
* Verified sha512 checksum was correct for source tarball
* Verified signature was correct for source tarball (not verified trust)
* Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
* Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
non-blocker for release as
https://issues.apache.org/jira/browse/HADOOP-19556
* Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
passing against Amazon S3 in eu-west-1
Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
tarihinde şunu yazdı:
> Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> included in the release yet, if the RC fails the vote, will include it in
> the next one.
>
> On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> anujmodi2011@gmail.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> Hi, Ahmar
>
>
> Here are the PRs for the fix raised for ABFS close() causing NPE raised on
> RC.
> PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> https://github.com/apache/hadoop/pull/7888>
> PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> https://github.com/apache/hadoop/pull/7889>
>
>
> Feel free to merge them as per the release timelines.
>
>
> Thanks
>
>
> On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> <ma...@cloudera.com.inva>lid>
> wrote:
>
>
> > just discovering as I do the third party compatibility testing of the
> > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> broken
> > -nothing to worry about though as its a new feature so not a regression
> >
> > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> <ma...@amazon.co.uk.inva>lid>
> > wrote:
> >
> > > Hey all,
> > >
> > > I've fixed the following issues:
> > > 1) Issues with changelog, release notes and landing page. Created a new
> > > build so they now include all the JIRA's that were missing before.
> > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > message when untarring
> > > 3) Previous issues with sha512 and tar not being gzipped.
> > >
> > > RC3 for Hadoop 3.4.2 is now available for voting.
> > >
> > > The RC is available at:
> > >
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > >
> > > The git tag is release-3.4.2-RC3, commit
> > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > >
> > > The maven artifacts are staged at:
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > >
> > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > >
> > >
> > > This release has been created off of branch-3.4.
> > >
> > > Thanks,
> > > Ahmar
> > >
> > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>
> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Since I have to regenerate the site and changelog, do I need to do
> the
> > > whole build again or is there an easier way to do that? (just by
> > > regenerating the site manually and updating the artifacts in SVN
> maybe?)
> > >
> > >
> > > I think just running the create-release[1] script could be an easy way
> > > while it does not provide options for building only site or source
> > > tarballs (now).
> > > Since the binary tarballs are containing affected CHANGELOG and
> > > RELEASENOTES,
> > > it would be nice to update the binary tarballs.
> > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > # could be ok if only documentation is updated.
> > > # Signing and checksums must be cared in that case.
> > >
> > >
> > > In addition, I got following warnings when I extracted the binary
> > tarballs
> > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > Ubuntu.::
> > >
> > >
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > >
> > >
> > > I guess the cause is modifying the tar files on OSX environment.
> > > It would be nice to fix too.
> > >
> > >
> > > [1]
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
> > > >
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > Thanks Masatake and Steve,
> > > >
> > > > Here's the PR for updating the top page of the site
> > > https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887> <
> https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> > we
> > > think needs to go in there, let me know, otherwise I will merge by
> > tomorrow.
> > > >
> > > > Since I have to regenerate the site and changelog, do I need to do
> the
> > > whole build again or is there an easier way to do that? (just by
> > > regenerating the site manually and updating the artifacts in SVN
> maybe?)
> > > >
> > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > >
> > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>>
> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thanks for the update, Ahmar
> > > >
> > > >
> > > > The binary artifacts basically looks good.
> > > > I have some comments about inconsistencies.
> > > >
> > > >
> > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > Version/s". I updated them.
> > > >
> > > >
> > > > Jira/Git commit message diff starting:
> > > ##############################################
> > > > Jira not present with version: 3.4.2. Commit:
> > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> #7816
> > > > ...
> > > > Jira not present with version: 3.4.2. Commit:
> > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > > version to 1.2.1 (#7807) (#7815)
> > > > Jira not present with version: 3.4.2. Commit:
> > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> LocalDirAllocator
> > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > ...
> > > > Jira not present with version: 3.4.2. Commit:
> > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > commons-cli
> > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > >
> > > >
> > > > - The top page of site documentation was not updated for 3.4.2.
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> >
> > > ;>
> > > >
> > > >
> > > > - Some release artifact are inconsistent as Steve pointed out.
> > > >
> > > >
> > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > >
> > > >
> > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > >> Hey all,
> > > >>
> > > >> Just a reminder about this. Would be great to get some votes in asap
> > and
> > > >> hopefully finalise 3.4.2.
> > > >>
> > > >> Thanks,
> > > >> Ahmar
> > > >>
> > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <ma...@apache.org>
> > <mailto:
> > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> ahmar@apache.org>>>>
> > > wrote:
> > > >>
> > > >>> Hey all,
> > > >>>
> > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > >>>
> > > >>> There are a couple of things to note:
> > > >>>
> > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > Chuang
> > > for
> > > >>> working on the fix.
> > > >>>
> > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> bundle
> > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > ~600MB,
> > > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> > to
> > > SVN.
> > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > >>>
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > >
> > > <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> >
> > > ;>,
> > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > updated
> > > >>> with these instructions.
> > > >>>
> > > >>>
> > > >>> The RC is available at:
> > > >>>
> > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > >>>
> > > >>> The git tag is release-3.4.2-RC2, commit
> > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > >>>
> > > >>> The maven artifacts are staged at:
> > > >>>
> > > >>>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> >
> > > ;>
> > > >>>
> > > >>>
> > > >>> You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > >>>
> > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > > >>>
> > > >>>
> > > >>> This release has been created off of branch-3.4.
> > > >>>
> > > >>> Thanks,
> > > >>> Ahmar
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> > >
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Erdoğan Özkoca <er...@gmail.com>.
+1 (non-binding)
* Verified sha512 checksum was correct for source tarball
* Verified signature was correct for source tarball (not verified trust)
* Built source code on Amazon-Linux 2 and OpenJDK 8 in Amazon EC2
* Verified S3A (hadoop-tools/hadoop-aws) unit tests passing except
ITestS3AContractOpen.testInputStreamReadNegativePosition test which is
non-blocker for release as
https://issues.apache.org/jira/browse/HADOOP-19556
* Verified S3A (hadoop-tools/hadoop-aws) integ tests with scale profile
passing against Amazon S3 in eu-west-1
Suhail, Ahmar <ah...@amazon.co.uk.invalid>, 21 Ağu 2025 Per, 10:52
tarihinde şunu yazdı:
> Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be
> included in the release yet, if the RC fails the vote, will include it in
> the next one.
>
> On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <mailto:
> anujmodi2011@gmail.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> Hi, Ahmar
>
>
> Here are the PRs for the fix raised for ABFS close() causing NPE raised on
> RC.
> PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <
> https://github.com/apache/hadoop/pull/7888>
> PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <
> https://github.com/apache/hadoop/pull/7889>
>
>
> Feel free to merge them as per the release timelines.
>
>
> Thanks
>
>
> On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva
> <ma...@cloudera.com.inva>lid>
> wrote:
>
>
> > just discovering as I do the third party compatibility testing of the
> > latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit
> broken
> > -nothing to worry about though as its a new feature so not a regression
> >
> > On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva
> <ma...@amazon.co.uk.inva>lid>
> > wrote:
> >
> > > Hey all,
> > >
> > > I've fixed the following issues:
> > > 1) Issues with changelog, release notes and landing page. Created a new
> > > build so they now include all the JIRA's that were missing before.
> > > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > message when untarring
> > > 3) Previous issues with sha512 and tar not being gzipped.
> > >
> > > RC3 for Hadoop 3.4.2 is now available for voting.
> > >
> > > The RC is available at:
> > >
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> > >
> > > The git tag is release-3.4.2-RC3, commit
> > > 84e8b89ee2ebe6923691205b9e171badde7a495c
> > >
> > > The maven artifacts are staged at:
> > >
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1443 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> > >
> > > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> > >
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> > >
> > >
> > > This release has been created off of branch-3.4.
> > >
> > > Thanks,
> > > Ahmar
> > >
> > > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>
> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > > Since I have to regenerate the site and changelog, do I need to do
> the
> > > whole build again or is there an easier way to do that? (just by
> > > regenerating the site manually and updating the artifacts in SVN
> maybe?)
> > >
> > >
> > > I think just running the create-release[1] script could be an easy way
> > > while it does not provide options for building only site or source
> > > tarballs (now).
> > > Since the binary tarballs are containing affected CHANGELOG and
> > > RELEASENOTES,
> > > it would be nice to update the binary tarballs.
> > > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > > # could be ok if only documentation is updated.
> > > # Signing and checksums must be cared in that case.
> > >
> > >
> > > In addition, I got following warnings when I extracted the binary
> > tarballs
> > > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > > Ubuntu.::
> > >
> > >
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.provenance'
> > > tar: Ignoring unknown extended header keyword
> > > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > >
> > >
> > > I guess the cause is modifying the tar files on OSX environment.
> > > It would be nice to fix too.
> > >
> > >
> > > [1]
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
> > > >
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > > Thanks Masatake and Steve,
> > > >
> > > > Here's the PR for updating the top page of the site
> > > https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> <
> > > https://github.com/apache/hadoop/pull/7887> <
> https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> > we
> > > think needs to go in there, let me know, otherwise I will merge by
> > tomorrow.
> > > >
> > > > Since I have to regenerate the site and changelog, do I need to do
> the
> > > whole build again or is there an easier way to do that? (just by
> > > regenerating the site manually and updating the artifacts in SVN
> maybe?)
> > > >
> > > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > > >
> > > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>
> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>>
> wrote:
> > > >
> > > >
> > > > CAUTION: This email originated from outside of the organization. Do
> not
> > > click links or open attachments unless you can confirm the sender and
> > know
> > > the content is safe.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thanks for the update, Ahmar
> > > >
> > > >
> > > > The binary artifacts basically looks good.
> > > > I have some comments about inconsistencies.
> > > >
> > > >
> > > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > > Version/s". I updated them.
> > > >
> > > >
> > > > Jira/Git commit message diff starting:
> > > ##############################################
> > > > Jira not present with version: 3.4.2. Commit:
> > > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS
> #7816
> > > > ...
> > > > Jira not present with version: 3.4.2. Commit:
> > > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > > version to 1.2.1 (#7807) (#7815)
> > > > Jira not present with version: 3.4.2. Commit:
> > > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554.
> LocalDirAllocator
> > > still doesn't always recover from directory deletion (#7683) (#7685)
> > > > ...
> > > > Jira not present with version: 3.4.2. Commit:
> > > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> > commons-cli
> > > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > > >
> > > >
> > > > - The top page of site documentation was not updated for 3.4.2.
> > > >
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > >
> > > <
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt
> >
> > > ;>
> > > >
> > > >
> > > > - Some release artifact are inconsistent as Steve pointed out.
> > > >
> > > >
> > > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > > >
> > > >
> > > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > > >
> > > >
> > > > Masatake Iwasaki
> > > >
> > > >
> > > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > > >> Hey all,
> > > >>
> > > >> Just a reminder about this. Would be great to get some votes in asap
> > and
> > > >> hopefully finalise 3.4.2.
> > > >>
> > > >> Thanks,
> > > >> Ahmar
> > > >>
> > > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <ma...@apache.org>
> > <mailto:
> > > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org
> <ma...@apache.org> <mailto:ahmar@apache.org <mailto:
> ahmar@apache.org>>>>
> > > wrote:
> > > >>
> > > >>> Hey all,
> > > >>>
> > > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > > >>>
> > > >>> There are a couple of things to note:
> > > >>>
> > > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> > Chuang
> > > for
> > > >>> working on the fix.
> > > >>>
> > > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK
> bundle
> > > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> > ~600MB,
> > > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> > to
> > > SVN.
> > > >>> For S3A, download SDK bundle v2.29.52 from:
> > > >>>
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> > >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > >
> > > <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>
> > > <
> > >
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt
> >
> > > ;>,
> > > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > > updated
> > > >>> with these instructions.
> > > >>>
> > > >>>
> > > >>> The RC is available at:
> > > >>>
> > > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > > >>>
> > > >>> The git tag is release-3.4.2-RC2, commit
> > > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > > >>>
> > > >>> The maven artifacts are staged at:
> > > >>>
> > > >>>
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> > <
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> > <
> > >
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt
> >
> > > ;>
> > > >>>
> > > >>>
> > > >>> You can find my public key
> (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > > at:
> > > >>>
> > > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > > >>>
> > > >>>
> > > >>> This release has been created off of branch-3.4.
> > > >>>
> > > >>> Thanks,
> > > >>> Ahmar
> > > >>>
> > > >>
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> > > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> > >
> >
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be included in the release yet, if the RC fails the vote, will include it in the next one.
On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888>
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889>
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> <mailto:
> > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be included in the release yet, if the RC fails the vote, will include it in the next one.
On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888>
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889>
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> <mailto:
> > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be included in the release yet, if the RC fails the vote, will include it in the next one.
On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888>
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889>
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> <mailto:
> > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Anuj, I've merged. Since I cut RC-3 yesterday, this won't be included in the release yet, if the RC fails the vote, will include it in the next one.
On 21/08/2025, 05:08, "Anuj Modi" <anujmodi2011@gmail.com <ma...@gmail.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888 <https://github.com/apache/hadoop/pull/7888>
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 <https://github.com/apache/hadoop/pull/7889>
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <stevel@cloudera.com.inva <ma...@cloudera.com.inva>lid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ahmarsu@amazon.co.uk.inva <ma...@amazon.co.uk.inva>lid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <
> > https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>
> > <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>
> <mailto:
> > ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Anuj Modi <an...@gmail.com>.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <mailto:
> > ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Anuj Modi <an...@gmail.com>.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <mailto:
> > ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Anuj Modi <an...@gmail.com>.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <mailto:
> > ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Anuj Modi <an...@gmail.com>.
Hi, Ahmar
Here are the PRs for the fix raised for ABFS close() causing NPE raised on
RC.
PR on branch-3.4.2: https://github.com/apache/hadoop/pull/7888
PR on branch-3.4: https://github.com/apache/hadoop/pull/7889
Feel free to merge them as per the release timelines.
Thanks
On Wed, Aug 20, 2025 at 11:24 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> just discovering as I do the third party compatibility testing of the
> latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
> -nothing to worry about though as its a new feature so not a regression
>
> On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
> wrote:
>
> > Hey all,
> >
> > I've fixed the following issues:
> > 1) Issues with changelog, release notes and landing page. Created a new
> > build so they now include all the JIRA's that were missing before.
> > 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> > unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> > message when untarring
> > 3) Previous issues with sha512 and tar not being gzipped.
> >
> > RC3 for Hadoop 3.4.2 is now available for voting.
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
> >
> > The git tag is release-3.4.2-RC3, commit
> > 84e8b89ee2ebe6923691205b9e171badde7a495c
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1443
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
> > On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> >
> >
> >
> >
> >
> >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> >
> > I think just running the create-release[1] script could be an easy way
> > while it does not provide options for building only site or source
> > tarballs (now).
> > Since the binary tarballs are containing affected CHANGELOG and
> > RELEASENOTES,
> > it would be nice to update the binary tarballs.
> > # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> > # could be ok if only documentation is updated.
> > # Signing and checksums must be cared in that case.
> >
> >
> > In addition, I got following warnings when I extracted the binary
> tarballs
> > (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> > Ubuntu.::
> >
> >
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.provenance'
> > tar: Ignoring unknown extended header keyword
> > 'LIBARCHIVE.xattr.com.apple.quarantine'
> >
> >
> > I guess the cause is modifying the tar files on OSX environment.
> > It would be nice to fix too.
> >
> >
> > [1]
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > <
> >
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> > >
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > > Thanks Masatake and Steve,
> > >
> > > Here's the PR for updating the top page of the site
> > https://github.com/apache/hadoop/pull/7887 <
> > https://github.com/apache/hadoop/pull/7887> , if there's anything else
> we
> > think needs to go in there, let me know, otherwise I will merge by
> tomorrow.
> > >
> > > Since I have to regenerate the site and changelog, do I need to do the
> > whole build again or is there an easier way to do that? (just by
> > regenerating the site manually and updating the artifacts in SVN maybe?)
> > >
> > > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> > hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> > >
> > > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> > <ma...@oss.nttdata.com>>> wrote:
> > >
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and
> know
> > the content is safe.
> > >
> > >
> > >
> > >
> > >
> > >
> > > Thanks for the update, Ahmar
> > >
> > >
> > > The binary artifacts basically looks good.
> > > I have some comments about inconsistencies.
> > >
> > >
> > > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> > Version/s". I updated them.
> > >
> > >
> > > Jira/Git commit message diff starting:
> > ##############################################
> > > Jira not present with version: 3.4.2. Commit:
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> > testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> > version to 1.2.1 (#7807) (#7815)
> > > Jira not present with version: 3.4.2. Commit:
> > 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> > still doesn't always recover from directory deletion (#7683) (#7685)
> > > ...
> > > Jira not present with version: 3.4.2. Commit:
> > 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade
> commons-cli
> > to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> > >
> > >
> > > - The top page of site documentation was not updated for 3.4.2.
> > >
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> >
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> > ;>
> > >
> > >
> > > - Some release artifact are inconsistent as Steve pointed out.
> > >
> > >
> > > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> > >
> > >
> > > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> > `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> > >
> > >
> > > Masatake Iwasaki
> > >
> > >
> > > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> > >> Hey all,
> > >>
> > >> Just a reminder about this. Would be great to get some votes in asap
> and
> > >> hopefully finalise 3.4.2.
> > >>
> > >> Thanks,
> > >> Ahmar
> > >>
> > >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org
> <mailto:
> > ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> > wrote:
> > >>
> > >>> Hey all,
> > >>>
> > >>> RC2 for Hadoop 3.4.2 is now available for voting.
> > >>>
> > >>> There are a couple of things to note:
> > >>>
> > >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu
> Chuang
> > for
> > >>> working on the fix.
> > >>>
> > >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now
> ~600MB,
> > >>> which makes the size of tar > 1GB, and it can no longer be uploaded
> to
> > SVN.
> > >>> For S3A, download SDK bundle v2.29.52 from:
> > >>>
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> <
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> >
> > <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > <
> >
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> > ;>,
> > >>> and drop it into /share/hadoop/common/lib. Release notes will be
> > updated
> > >>> with these instructions.
> > >>>
> > >>>
> > >>> The RC is available at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> > >>>
> > >>> The git tag is release-3.4.2-RC2, commit
> > >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> > >>>
> > >>> The maven artifacts are staged at:
> > >>>
> > >>>
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> <
> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> > ;>
> > >>>
> > >>>
> > >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> > at:
> > >>>
> > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> > >>>
> > >>>
> > >>> This release has been created off of branch-3.4.
> > >>>
> > >>> Thanks,
> > >>> Ahmar
> > >>>
> > >>
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
just discovering as I do the third party compatibility testing of the
latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
-nothing to worry about though as its a new feature so not a regression
On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new
> build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source
> tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and
> RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1]
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > Thanks Masatake and Steve,
> >
> > Here's the PR for updating the top page of the site
> https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> , if there's anything else we
> think needs to go in there, let me know, otherwise I will merge by tomorrow.
> >
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> >
> > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> >
> >
> >
> > Thanks for the update, Ahmar
> >
> >
> > The binary artifacts basically looks good.
> > I have some comments about inconsistencies.
> >
> >
> > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> Version/s". I updated them.
> >
> >
> > Jira/Git commit message diff starting:
> ##############################################
> > Jira not present with version: 3.4.2. Commit:
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> version to 1.2.1 (#7807) (#7815)
> > Jira not present with version: 3.4.2. Commit:
> 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> still doesn't always recover from directory deletion (#7683) (#7685)
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli
> to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> >
> >
> > - The top page of site documentation was not updated for 3.4.2.
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> ;>
> >
> >
> > - Some release artifact are inconsistent as Steve pointed out.
> >
> >
> > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> >
> >
> > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> >> Hey all,
> >>
> >> Just a reminder about this. Would be great to get some votes in asap and
> >> hopefully finalise 3.4.2.
> >>
> >> Thanks,
> >> Ahmar
> >>
> >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <mailto:
> ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> wrote:
> >>
> >>> Hey all,
> >>>
> >>> RC2 for Hadoop 3.4.2 is now available for voting.
> >>>
> >>> There are a couple of things to note:
> >>>
> >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> >>> working on the fix.
> >>>
> >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> >>> which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> >>> For S3A, download SDK bundle v2.29.52 from:
> >>>
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>,
> >>> and drop it into /share/hadoop/common/lib. Release notes will be
> updated
> >>> with these instructions.
> >>>
> >>>
> >>> The RC is available at:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> >>>
> >>> The git tag is release-3.4.2-RC2, commit
> >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >>>
> >>> The maven artifacts are staged at:
> >>>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> >>>
> >>>
> >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> >>>
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> >>>
> >>>
> >>> This release has been created off of branch-3.4.
> >>>
> >>> Thanks,
> >>> Ahmar
> >>>
> >>
> >
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for testing and voting, and apologies for not creating a separate thread for RC3, will keep that in mind next time.
The voting summary is:
+5 binding (Ayush, Steve, He Xiaoqiao, Mukund, Masatake)
+3 non-binding (Erdogan, Dongjoon, Cheng Pan)
With 5 binding votes, the Hadoop release 3.4.2 has passed.
I will now process with the next steps to finalise it.
Thanks again to everyone for the help!
Ahmar
On 20/08/2025, 16:14, "Suhail, Ahmar" <ahmarsu@amazon.co.uk.INVA <ma...@amazon.co.uk.INVA>LID> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt;>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt;>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt;>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
just discovering as I do the third party compatibility testing of the
latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
-nothing to worry about though as its a new feature so not a regression
On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new
> build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source
> tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and
> RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1]
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > Thanks Masatake and Steve,
> >
> > Here's the PR for updating the top page of the site
> https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> , if there's anything else we
> think needs to go in there, let me know, otherwise I will merge by tomorrow.
> >
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> >
> > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> >
> >
> >
> > Thanks for the update, Ahmar
> >
> >
> > The binary artifacts basically looks good.
> > I have some comments about inconsistencies.
> >
> >
> > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> Version/s". I updated them.
> >
> >
> > Jira/Git commit message diff starting:
> ##############################################
> > Jira not present with version: 3.4.2. Commit:
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> version to 1.2.1 (#7807) (#7815)
> > Jira not present with version: 3.4.2. Commit:
> 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> still doesn't always recover from directory deletion (#7683) (#7685)
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli
> to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> >
> >
> > - The top page of site documentation was not updated for 3.4.2.
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> ;>
> >
> >
> > - Some release artifact are inconsistent as Steve pointed out.
> >
> >
> > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> >
> >
> > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> >> Hey all,
> >>
> >> Just a reminder about this. Would be great to get some votes in asap and
> >> hopefully finalise 3.4.2.
> >>
> >> Thanks,
> >> Ahmar
> >>
> >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <mailto:
> ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> wrote:
> >>
> >>> Hey all,
> >>>
> >>> RC2 for Hadoop 3.4.2 is now available for voting.
> >>>
> >>> There are a couple of things to note:
> >>>
> >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> >>> working on the fix.
> >>>
> >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> >>> which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> >>> For S3A, download SDK bundle v2.29.52 from:
> >>>
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>,
> >>> and drop it into /share/hadoop/common/lib. Release notes will be
> updated
> >>> with these instructions.
> >>>
> >>>
> >>> The RC is available at:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> >>>
> >>> The git tag is release-3.4.2-RC2, commit
> >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >>>
> >>> The maven artifacts are staged at:
> >>>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> >>>
> >>>
> >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> >>>
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> >>>
> >>>
> >>> This release has been created off of branch-3.4.
> >>>
> >>> Thanks,
> >>> Ahmar
> >>>
> >>
> >
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for testing and voting, and apologies for not creating a separate thread for RC3, will keep that in mind next time.
The voting summary is:
+5 binding (Ayush, Steve, He Xiaoqiao, Mukund, Masatake)
+3 non-binding (Erdogan, Dongjoon, Cheng Pan)
With 5 binding votes, the Hadoop release 3.4.2 has passed.
I will now process with the next steps to finalise it.
Thanks again to everyone for the help!
Ahmar
On 20/08/2025, 16:14, "Suhail, Ahmar" <ahmarsu@amazon.co.uk.INVA <ma...@amazon.co.uk.INVA>LID> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt;>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt;>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt;>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
just discovering as I do the third party compatibility testing of the
latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
-nothing to worry about though as its a new feature so not a regression
On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new
> build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source
> tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and
> RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1]
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > Thanks Masatake and Steve,
> >
> > Here's the PR for updating the top page of the site
> https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> , if there's anything else we
> think needs to go in there, let me know, otherwise I will merge by tomorrow.
> >
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> >
> > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> >
> >
> >
> > Thanks for the update, Ahmar
> >
> >
> > The binary artifacts basically looks good.
> > I have some comments about inconsistencies.
> >
> >
> > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> Version/s". I updated them.
> >
> >
> > Jira/Git commit message diff starting:
> ##############################################
> > Jira not present with version: 3.4.2. Commit:
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> version to 1.2.1 (#7807) (#7815)
> > Jira not present with version: 3.4.2. Commit:
> 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> still doesn't always recover from directory deletion (#7683) (#7685)
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli
> to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> >
> >
> > - The top page of site documentation was not updated for 3.4.2.
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> ;>
> >
> >
> > - Some release artifact are inconsistent as Steve pointed out.
> >
> >
> > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> >
> >
> > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> >> Hey all,
> >>
> >> Just a reminder about this. Would be great to get some votes in asap and
> >> hopefully finalise 3.4.2.
> >>
> >> Thanks,
> >> Ahmar
> >>
> >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <mailto:
> ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> wrote:
> >>
> >>> Hey all,
> >>>
> >>> RC2 for Hadoop 3.4.2 is now available for voting.
> >>>
> >>> There are a couple of things to note:
> >>>
> >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> >>> working on the fix.
> >>>
> >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> >>> which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> >>> For S3A, download SDK bundle v2.29.52 from:
> >>>
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>,
> >>> and drop it into /share/hadoop/common/lib. Release notes will be
> updated
> >>> with these instructions.
> >>>
> >>>
> >>> The RC is available at:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> >>>
> >>> The git tag is release-3.4.2-RC2, commit
> >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >>>
> >>> The maven artifacts are staged at:
> >>>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> >>>
> >>>
> >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> >>>
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> >>>
> >>>
> >>> This release has been created off of branch-3.4.
> >>>
> >>> Thanks,
> >>> Ahmar
> >>>
> >>
> >
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
just discovering as I do the third party compatibility testing of the
latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken
-nothing to worry about though as its a new feature so not a regression
On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar <ah...@amazon.co.uk.invalid>
wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new
> build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring
> unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
>
>
>
>
>
>
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source
> tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and
> RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on
> Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword
> 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1]
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> <
> https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
> >
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> > Thanks Masatake and Steve,
> >
> > Here's the PR for updating the top page of the site
> https://github.com/apache/hadoop/pull/7887 <
> https://github.com/apache/hadoop/pull/7887> , if there's anything else we
> think needs to go in there, let me know, otherwise I will merge by tomorrow.
> >
> > Since I have to regenerate the site and changelog, do I need to do the
> whole build again or is there an easier way to do that? (just by
> regenerating the site manually and updating the artifacts in SVN maybe?)
> >
> > I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and
> hadoop-3.4.2-aarch64-lean.tar.sha512 now.
> >
> > On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com
> <ma...@oss.nttdata.com>>> wrote:
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and know
> the content is safe.
> >
> >
> >
> >
> >
> >
> > Thanks for the update, Ahmar
> >
> >
> > The binary artifacts basically looks good.
> > I have some comments about inconsistencies.
> >
> >
> > - Some JIRAs of additional commits did not have 3.4.2 as "Fix
> Version/s". I updated them.
> >
> >
> > Jira/Git commit message diff starting:
> ##############################################
> > Jira not present with version: 3.4.2. Commit:
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A:
> testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL
> version to 1.2.1 (#7807) (#7815)
> > Jira not present with version: 3.4.2. Commit:
> 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator
> still doesn't always recover from directory deletion (#7683) (#7685)
> > ...
> > Jira not present with version: 3.4.2. Commit:
> 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli
> to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
> >
> >
> > - The top page of site documentation was not updated for 3.4.2.
> >
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> <
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
> ;>
> >
> >
> > - Some release artifact are inconsistent as Steve pointed out.
> >
> >
> > - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
> >
> >
> > - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be
> `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
> >
> >
> > Masatake Iwasaki
> >
> >
> > On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> >> Hey all,
> >>
> >> Just a reminder about this. Would be great to get some votes in asap and
> >> hopefully finalise 3.4.2.
> >>
> >> Thanks,
> >> Ahmar
> >>
> >> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <mailto:
> ahmar@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>
> wrote:
> >>
> >>> Hey all,
> >>>
> >>> RC2 for Hadoop 3.4.2 is now available for voting.
> >>>
> >>> There are a couple of things to note:
> >>>
> >>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> >>> working on the fix.
> >>>
> >>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> >>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> >>> which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> >>> For S3A, download SDK bundle v2.29.52 from:
> >>>
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> <
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>
> ;>,
> >>> and drop it into /share/hadoop/common/lib. Release notes will be
> updated
> >>> with these instructions.
> >>>
> >>>
> >>> The RC is available at:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
> >>>
> >>> The git tag is release-3.4.2-RC2, commit
> >>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >>>
> >>> The maven artifacts are staged at:
> >>>
> >>>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441> <
> https://repository.apache.org/content/repositories/orgapachehadoop-1441>
> ;>
> >>>
> >>>
> >>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98)
> at:
> >>>
> >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
> >>>
> >>>
> >>> This release has been created off of branch-3.4.
> >>>
> >>> Thanks,
> >>> Ahmar
> >>>
> >>
> >
> >
> >
> >
> >
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
+1 (binding)
+ verified checksums and signatures of src and site tarballs.
+ built from source tarball with native profile enabled on
Rocky Linux 9 (x86_64 and aarch64).
+ deployed pseudo cluster with and without kerberos enabled.
+ ran some example jobs.
+ ran some simple query against httpfs.
+ skimmed the contents of pre-built site documentation.
Thanks for the great work, Ahmar.
Masatake Iwasaki
On 8/21/2025 12:13 AM, Suhail, Ahmar wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
>> Thanks Masatake and Steve,
>>
>> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>>
>> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>>
>> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>>
>>
>> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>>
>>
>>
>>
>>
>>
>> Thanks for the update, Ahmar
>>
>>
>> The binary artifacts basically looks good.
>> I have some comments about inconsistencies.
>>
>>
>> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>>
>>
>> Jira/Git commit message diff starting: ##############################################
>> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
>> ...
>> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
>> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
>> ...
>> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>>
>>
>> - The top page of site documentation was not updated for 3.4.2.
>> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>>
>>
>> - Some release artifact are inconsistent as Steve pointed out.
>>
>>
>> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>>
>>
>> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>>
>>
>> Masatake Iwasaki
>>
>>
>> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>>> Hey all,
>>>
>>> Just a reminder about this. Would be great to get some votes in asap and
>>> hopefully finalise 3.4.2.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>>
>>>> Hey all,
>>>>
>>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>>
>>>> There are a couple of things to note:
>>>>
>>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>>> working on the fix.
>>>>
>>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>>> For S3A, download SDK bundle v2.29.52 from:
>>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>>> with these instructions.
>>>>
>>>>
>>>> The RC is available at:
>>>>
>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>>
>>>> The git tag is release-3.4.2-RC2, commit
>>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>>
>>>> The maven artifacts are staged at:
>>>>
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>>
>>>>
>>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>>
>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>>
>>>>
>>>> This release has been created off of branch-3.4.
>>>>
>>>> Thanks,
>>>> Ahmar
>>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
+1 (binding)
+ verified checksums and signatures of src and site tarballs.
+ built from source tarball with native profile enabled on
Rocky Linux 9 (x86_64 and aarch64).
+ deployed pseudo cluster with and without kerberos enabled.
+ ran some example jobs.
+ ran some simple query against httpfs.
+ skimmed the contents of pre-built site documentation.
Thanks for the great work, Ahmar.
Masatake Iwasaki
On 8/21/2025 12:13 AM, Suhail, Ahmar wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
>> Thanks Masatake and Steve,
>>
>> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>>
>> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>>
>> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>>
>>
>> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>>
>>
>>
>>
>>
>>
>> Thanks for the update, Ahmar
>>
>>
>> The binary artifacts basically looks good.
>> I have some comments about inconsistencies.
>>
>>
>> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>>
>>
>> Jira/Git commit message diff starting: ##############################################
>> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
>> ...
>> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
>> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
>> ...
>> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>>
>>
>> - The top page of site documentation was not updated for 3.4.2.
>> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>>
>>
>> - Some release artifact are inconsistent as Steve pointed out.
>>
>>
>> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>>
>>
>> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>>
>>
>> Masatake Iwasaki
>>
>>
>> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>>> Hey all,
>>>
>>> Just a reminder about this. Would be great to get some votes in asap and
>>> hopefully finalise 3.4.2.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>>
>>>> Hey all,
>>>>
>>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>>
>>>> There are a couple of things to note:
>>>>
>>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>>> working on the fix.
>>>>
>>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>>> For S3A, download SDK bundle v2.29.52 from:
>>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>>> with these instructions.
>>>>
>>>>
>>>> The RC is available at:
>>>>
>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>>
>>>> The git tag is release-3.4.2-RC2, commit
>>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>>
>>>> The maven artifacts are staged at:
>>>>
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>>
>>>>
>>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>>
>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>>
>>>>
>>>> This release has been created off of branch-3.4.
>>>>
>>>> Thanks,
>>>> Ahmar
>>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thank you everyone for testing and voting, and apologies for not creating a separate thread for RC3, will keep that in mind next time.
The voting summary is:
+5 binding (Ayush, Steve, He Xiaoqiao, Mukund, Masatake)
+3 non-binding (Erdogan, Dongjoon, Cheng Pan)
With 5 binding votes, the Hadoop release 3.4.2 has passed.
I will now process with the next steps to finalise it.
Thanks again to everyone for the help!
Ahmar
On 20/08/2025, 16:14, "Suhail, Ahmar" <ahmarsu@amazon.co.uk.INVA <ma...@amazon.co.uk.INVA>LID> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/>
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443 <https://repository.apache.org/content/repositories/orgapachehadoop-1443>
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release> <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887> <https://github.com/apache/hadoop/pull/7887>> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm&gt;>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>> <mailto:ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52&gt;>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/&gt;>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>> <https://repository.apache.org/content/repositories/orgapachehadoop-1441&gt;>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS&gt;>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org <ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org <ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
+1 (binding)
+ verified checksums and signatures of src and site tarballs.
+ built from source tarball with native profile enabled on
Rocky Linux 9 (x86_64 and aarch64).
+ deployed pseudo cluster with and without kerberos enabled.
+ ran some example jobs.
+ ran some simple query against httpfs.
+ skimmed the contents of pre-built site documentation.
Thanks for the great work, Ahmar.
Masatake Iwasaki
On 8/21/2025 12:13 AM, Suhail, Ahmar wrote:
> Hey all,
>
> I've fixed the following issues:
> 1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
> 2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
> 3) Previous issues with sha512 and tar not being gzipped.
>
> RC3 for Hadoop 3.4.2 is now available for voting.
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
>
> The git tag is release-3.4.2-RC3, commit
> 84e8b89ee2ebe6923691205b9e171badde7a495c
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1443
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
> On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
>
> I think just running the create-release[1] script could be an easy way
> while it does not provide options for building only site or source tarballs (now).
> Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
> it would be nice to update the binary tarballs.
> # Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
> # could be ok if only documentation is updated.
> # Signing and checksums must be cared in that case.
>
>
> In addition, I got following warnings when I extracted the binary tarballs
> (hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
>
>
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
> tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
>
>
> I guess the cause is modifying the tar files on OSX environment.
> It would be nice to fix too.
>
>
> [1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
>
>
> Masatake Iwasaki
>
>
> On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
>> Thanks Masatake and Steve,
>>
>> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>>
>> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>>
>> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>>
>> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>>
>>
>> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>>
>>
>>
>>
>>
>>
>> Thanks for the update, Ahmar
>>
>>
>> The binary artifacts basically looks good.
>> I have some comments about inconsistencies.
>>
>>
>> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>>
>>
>> Jira/Git commit message diff starting: ##############################################
>> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
>> ...
>> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
>> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
>> ...
>> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>>
>>
>> - The top page of site documentation was not updated for 3.4.2.
>> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>>
>>
>> - Some release artifact are inconsistent as Steve pointed out.
>>
>>
>> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>>
>>
>> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>>
>>
>> Masatake Iwasaki
>>
>>
>> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>>> Hey all,
>>>
>>> Just a reminder about this. Would be great to get some votes in asap and
>>> hopefully finalise 3.4.2.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>>
>>>> Hey all,
>>>>
>>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>>
>>>> There are a couple of things to note:
>>>>
>>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>>> working on the fix.
>>>>
>>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>>> For S3A, download SDK bundle v2.29.52 from:
>>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>>> with these instructions.
>>>>
>>>>
>>>> The RC is available at:
>>>>
>>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>>
>>>> The git tag is release-3.4.2-RC2, commit
>>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>>
>>>> The maven artifacts are staged at:
>>>>
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>>
>>>>
>>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>>
>>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>>
>>>>
>>>> This release has been created off of branch-3.4.
>>>>
>>>> Thanks,
>>>> Ahmar
>>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Hey all,
I've fixed the following issues:
1) Issues with changelog, release notes and landing page. Created a new build so they now include all the JIRA's that were missing before.
2) Create the lean tar on ubuntu so you won't get the tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine' message when untarring
3) Previous issues with sha512 and tar not being gzipped.
RC3 for Hadoop 3.4.2 is now available for voting.
The RC is available at:
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC3/
The git tag is release-3.4.2-RC3, commit
84e8b89ee2ebe6923691205b9e171badde7a495c
The maven artifacts are staged at:
https://repository.apache.org/content/repositories/orgapachehadoop-1443
You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
This release has been created off of branch-3.4.
Thanks,
Ahmar
On 20/08/2025, 09:25, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release <https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release>
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 <https://github.com/apache/hadoop/pull/7887> , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com> <mailto:iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm> <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org> <mailto:ahmar@apache.org <ma...@apache.org>>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52> <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/> <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441> <https://repository.apache.org/content/repositories/orgapachehadoop-1441>>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS> <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I think just running the create-release[1] script could be an easy way
while it does not provide options for building only site or source tarballs (now).
Since the binary tarballs are containing affected CHANGELOG and RELEASENOTES,
it would be nice to update the binary tarballs.
# Swapping the CHANGELOG and RELEASENOTES of existing binary tarballs
# could be ok if only documentation is updated.
# Signing and checksums must be cared in that case.
In addition, I got following warnings when I extracted the binary tarballs
(hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.gz) on Ubuntu.::
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.quarantine'
I guess the cause is modifying the tar files on OSX environment.
It would be nice to fix too.
[1] https://github.com/apache/hadoop/blob/branch-3.4.2/dev-support/bin/create-release
Masatake Iwasaki
On 8/19/2025 10:28 PM, Suhail, Ahmar wrote:
> Thanks Masatake and Steve,
>
> Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
>
> Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
>
> I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
>
> On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
>
>
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
>
>
>
> Thanks for the update, Ahmar
>
>
> The binary artifacts basically looks good.
> I have some comments about inconsistencies.
>
>
> - Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
>
>
> Jira/Git commit message diff starting: ##############################################
> Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
> ...
> Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
> Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
> ...
> Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
>
>
> - The top page of site documentation was not updated for 3.4.2.
> https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
>
>
> - Some release artifact are inconsistent as Steve pointed out.
>
>
> - `hadoop-3.4.2-lean.tar.gz` is not gzipped.
>
>
> - `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
>
>
> Masatake Iwasaki
>
>
> On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
>> Hey all,
>>
>> Just a reminder about this. Would be great to get some votes in asap and
>> hopefully finalise 3.4.2.
>>
>> Thanks,
>> Ahmar
>>
>> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>>
>>> Hey all,
>>>
>>> RC2 for Hadoop 3.4.2 is now available for voting.
>>>
>>> There are a couple of things to note:
>>>
>>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>>> working on the fix.
>>>
>>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>>> For S3A, download SDK bundle v2.29.52 from:
>>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>>> with these instructions.
>>>
>>>
>>> The RC is available at:
>>>
>>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>>
>>> The git tag is release-3.4.2-RC2, commit
>>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>>
>>> The maven artifacts are staged at:
>>>
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>>
>>>
>>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>>
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>>
>>>
>>> This release has been created off of branch-3.4.
>>>
>>> Thanks,
>>> Ahmar
>>>
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Masatake and Steve,
Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Masatake and Steve,
Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Masatake and Steve,
Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by "Suhail, Ahmar" <ah...@amazon.co.uk.INVALID>.
Thanks Masatake and Steve,
Here's the PR for updating the top page of the site https://github.com/apache/hadoop/pull/7887 , if there's anything else we think needs to go in there, let me know, otherwise I will merge by tomorrow.
Since I have to regenerate the site and changelog, do I need to do the whole build again or is there an easier way to do that? (just by regenerating the site manually and updating the artifacts in SVN maybe?)
I will update the artifacts for the hadoop-3.4.2-lean.tar.gz and hadoop-3.4.2-aarch64-lean.tar.sha512 now.
On 19/08/2025, 08:26, "Masatake Iwasaki" <iwasakims@oss.nttdata.com <ma...@oss.nttdata.com>> wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm <https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm>
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ahmar@apache.org <ma...@apache.org>> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52 <https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52>,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/ <https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/>
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441 <https://repository.apache.org/content/repositories/orgapachehadoop-1441>
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS <https://dist.apache.org/repos/dist/release/hadoop/common/KEYS>
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Xiaoqiao He <he...@apache.org>.
Thanks Ahmar for your work on this. Will check it ASAP.
Best Regards,
- He Xiaoqiao
On Thu, Aug 14, 2025 at 5:14 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
sorry, been behind on this.
-1 packaging related.
that hadoop-3.4.2-lean.tar.gz file is actually a tar file. Took me a while
to work this down as I just assumed my ant build script was at fault.
file downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz
downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz: POSIX tar archive
I have verified that copying the right aws sdk version into tools/lib puts
it on the classpath.
Also found an NPE in the sequence new AzureBlobFileSystem().close() (i.e.
no Initialize)
Filed HADOOP-19650 for that
Also, there's no sha512 of the ant lean stuff.
Can't get
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
to downloads/incoming/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
Actual binaries, maven artifacts, good.
Parquet tests are failing with that buffer release logic which we have now
made optional in our code (HADOOP`18296). I'm wondering if we should have
parquet make vector read optional rather than on by default -but advise
turning on for all cloud stores.
Really unsure what to do there.
On Thu, 14 Aug 2025 at 10:13, Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
sorry, been behind on this.
-1 packaging related.
that hadoop-3.4.2-lean.tar.gz file is actually a tar file. Took me a while
to work this down as I just assumed my ant build script was at fault.
file downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz
downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz: POSIX tar archive
I have verified that copying the right aws sdk version into tools/lib puts
it on the classpath.
Also found an NPE in the sequence new AzureBlobFileSystem().close() (i.e.
no Initialize)
Filed HADOOP-19650 for that
Also, there's no sha512 of the ant lean stuff.
Can't get
https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
to downloads/incoming/hadoop-3.4.2-aarch64-lean.tar.gz.sha512
Actual binaries, maven artifacts, good.
Parquet tests are failing with that buffer release logic which we have now
made optional in our code (HADOOP`18296). I'm wondering if we should have
parquet make vector read optional rather than on by default -but advise
turning on for all cloud stores.
Really unsure what to do there.
On Thu, 14 Aug 2025 at 10:13, Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
> > Hey all,
> >
> > RC2 for Hadoop 3.4.2 is now available for voting.
> >
> > There are a couple of things to note:
> >
> > 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang
> for
> > working on the fix.
> >
> > 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> > from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> > which makes the size of tar > 1GB, and it can no longer be uploaded to
> SVN.
> > For S3A, download SDK bundle v2.29.52 from:
> > https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52
> ,
> > and drop it into /share/hadoop/common/lib. Release notes will be updated
> > with these instructions.
> >
> >
> > The RC is available at:
> >
> > https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
> >
> > The git tag is release-3.4.2-RC2, commit
> > e1c0dee881820a4d834ec4a4d2c70d0d953bb933
> >
> > The maven artifacts are staged at:
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1441
> >
> >
> > You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
> >
> > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >
> >
> > This release has been created off of branch-3.4.
> >
> > Thanks,
> > Ahmar
> >
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Masatake Iwasaki <iw...@oss.nttdata.com>.
Thanks for the update, Ahmar
The binary artifacts basically looks good.
I have some comments about inconsistencies.
- Some JIRAs of additional commits did not have 3.4.2 as "Fix Version/s". I updated them.
Jira/Git commit message diff starting: ##############################################
Jira not present with version: 3.4.2. Commit: e1c0dee881820a4d834ec4a4d2c70d0d953bb933 HADOOP-19627. S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS #7816
...
Jira not present with version: 3.4.2. Commit: 6a4ea9b1155aefb7732b700d8d7154f977141e39 HADOOP-19626. S3A: Updates AAL version to 1.2.1 (#7807) (#7815)
Jira not present with version: 3.4.2. Commit: 9677363829c21adeb2ce41f93059ab65fc115862 HADOOP-19554. LocalDirAllocator still doesn't always recover from directory deletion (#7683) (#7685)
...
Jira not present with version: 3.4.2. Commit: 57d821e6aad23b604833272071c8481567363e63 HADOOP-19143. Upgrade commons-cli to 1.9.0. (#7126) (#7146) Contributed by Min Yan.
- The top page of site documentation was not updated for 3.4.2.
https://github.com/apache/hadoop/blob/release-3.4.2-RC2/hadoop-project/src/site/markdown/index.md.vm
- Some release artifact are inconsistent as Steve pointed out.
- `hadoop-3.4.2-lean.tar.gz` is not gzipped.
- `hadoop-3.4.2-aarch64-lean.tar.sha512` should be `hadoop-3.4.2-aarch64-lean.tar.gz.sha512`
Masatake Iwasaki
On 8/14/2025 6:12 PM, Ahmar Suhail wrote:
> Hey all,
>
> Just a reminder about this. Would be great to get some votes in asap and
> hopefully finalise 3.4.2.
>
> Thanks,
> Ahmar
>
> On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
>
>> Hey all,
>>
>> RC2 for Hadoop 3.4.2 is now available for voting.
>>
>> There are a couple of things to note:
>>
>> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
>> working on the fix.
>>
>> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
>> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
>> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
>> For S3A, download SDK bundle v2.29.52 from:
>> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
>> and drop it into /share/hadoop/common/lib. Release notes will be updated
>> with these instructions.
>>
>>
>> The RC is available at:
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>>
>> The git tag is release-3.4.2-RC2, commit
>> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>>
>> The maven artifacts are staged at:
>>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>>
>>
>> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>>
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>>
>> This release has been created off of branch-3.4.
>>
>> Thanks,
>> Ahmar
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ahmar Suhail <ah...@apache.org>.
Hey all,
Just a reminder about this. Would be great to get some votes in asap and
hopefully finalise 3.4.2.
Thanks,
Ahmar
On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> RC2 for Hadoop 3.4.2 is now available for voting.
>
> There are a couple of things to note:
>
> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
> working on the fix.
>
> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
> For S3A, download SDK bundle v2.29.52 from:
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
> and drop it into /share/hadoop/common/lib. Release notes will be updated
> with these instructions.
>
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>
> The git tag is release-3.4.2-RC2, commit
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ahmar Suhail <ah...@apache.org>.
Hey all,
Just a reminder about this. Would be great to get some votes in asap and
hopefully finalise 3.4.2.
Thanks,
Ahmar
On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> RC2 for Hadoop 3.4.2 is now available for voting.
>
> There are a couple of things to note:
>
> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
> working on the fix.
>
> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
> For S3A, download SDK bundle v2.29.52 from:
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
> and drop it into /share/hadoop/common/lib. Release notes will be updated
> with these instructions.
>
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>
> The git tag is release-3.4.2-RC2, commit
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ahmar Suhail <ah...@apache.org>.
Hey all,
Just a reminder about this. Would be great to get some votes in asap and
hopefully finalise 3.4.2.
Thanks,
Ahmar
On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> RC2 for Hadoop 3.4.2 is now available for voting.
>
> There are a couple of things to note:
>
> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
> working on the fix.
>
> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
> For S3A, download SDK bundle v2.29.52 from:
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
> and drop it into /share/hadoop/common/lib. Release notes will be updated
> with these instructions.
>
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>
> The git tag is release-3.4.2-RC2, commit
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>
Re: [VOTE] Release Apache Hadoop 3.4.2
Posted by Ahmar Suhail <ah...@apache.org>.
Hey all,
Just a reminder about this. Would be great to get some votes in asap and
hopefully finalise 3.4.2.
Thanks,
Ahmar
On Thu, Aug 7, 2025 at 9:35 PM Ahmar Suhail <ah...@apache.org> wrote:
> Hey all,
>
> RC2 for Hadoop 3.4.2 is now available for voting.
>
> There are a couple of things to note:
>
> 1/ We have ARM artifacts again! Thanks to Masatake and Wei-Chiu Chuang for
> working on the fix.
>
> 2/ Relevant for anyone testing S3A: We've removed the AWS SDK bundle
> from hadoop-3.4.2.tar.gz. This is because the SDK bundle is now ~600MB,
> which makes the size of tar > 1GB, and it can no longer be uploaded to SVN.
> For S3A, download SDK bundle v2.29.52 from:
> https://mvnrepository.com/artifact/software.amazon.awssdk/bundle/2.29.52,
> and drop it into /share/hadoop/common/lib. Release notes will be updated
> with these instructions.
>
>
> The RC is available at:
>
> https://dist.apache.org/repos/dist/dev/hadoop/3.4.2-RC2/
>
> The git tag is release-3.4.2-RC2, commit
> e1c0dee881820a4d834ec4a4d2c70d0d953bb933
>
> The maven artifacts are staged at:
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1441
>
>
> You can find my public key (02085AFB652F796A3B01D11FD737A6F52281FA98) at:
>
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> This release has been created off of branch-3.4.
>
> Thanks,
> Ahmar
>