You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ahmarsuhail (via GitHub)" <gi...@apache.org> on 2025/08/19 13:18:25 UTC
[PR] HADOOP-19657. Updates site landing page. [hadoop]
ahmarsuhail opened a new pull request, #7887:
URL: https://github.com/apache/hadoop/pull/7887
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
Updates landing page with 3.4.2 changes.
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "ahmarsuhail (via GitHub)" <gi...@apache.org>.
ahmarsuhail merged PR #7887:
URL: https://github.com/apache/hadoop/pull/7887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "anujmodi2021 (via GitHub)" <gi...@apache.org>.
anujmodi2021 commented on code in PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#discussion_r2286959916
##########
hadoop-project/src/site/markdown/index.md.vm:
##########
@@ -23,75 +23,33 @@ Overview of Changes
Users are encouraged to read the full set of release notes.
This page provides an overview of the major changes.
-Bulk Delete API
-----------------------------------------
-
-[HADOOP-18679](https://issues.apache.org/jira/browse/HADOOP-18679) Bulk Delete API.
-
-This release provides an API to perform bulk delete of files/objects
-in an object store or filesystem.
-
New binary distribution
-----------------------
-[HADOOP-19083](https://issues.apache.org/jira/browse/HADOOP-19083) provide hadoop binary tarball without aws v2 sdk
-
-Hadoop has added a new variant of the binary distribution tarball, labeled with "lean" in the file
-name. This tarball excludes the full AWS SDK v2 bundle, resulting in approximately 50% reduction in
-file size.
+As of v3.4.2, Hadoop will only be distributed with a lean tarball, which excludes the full AWS SDK v2 bundle to reduce
+overall file size. This release has been tested with AWS SDK v2 2.29.52, which can be downloaded from Maven
+[here](https://mvnrepository.com/artifact/software.amazon.awssdk/aws-sdk-java/2.29.52).
S3A improvements
----------------
**Improvement**
-[HADOOP-18886](https://issues.apache.org/jira/browse/HADOOP-18886) S3A: AWS SDK V2 Migration: stabilization and S3Express
-
-This release completes stabilization efforts on the AWS SDK v2 migration and support of Amazon S3
-Express One Zone storage. S3 Select is no longer supported.
-
-[HADOOP-18993](https://issues.apache.org/jira/browse/HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)
-
-This introduces configuration property `fs.s3a.classloader.isolation`, which defaults to `true`.
-Set to `false` to disable S3A classloader isolation, which can be useful for installing custom
-credential providers in user-provided jars.
-
-[HADOOP-19047](https://issues.apache.org/jira/browse/HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits
-
-The S3A magic committer now supports configuration property
-`fs.s3a.committer.magic.track.commits.in.memory.enabled`. Set this to `true` to track commits in
-memory instead of on the file system, which reduces the number of remote calls.
+[HADOOP-19363](https://issues.apache.org/jira/browse/HADOOP-19363) S3A: Support analytics-accelerator-s3 input streams
+for parquet read performance.
-[HADOOP-19161](https://issues.apache.org/jira/browse/HADOOP-19161) S3A: option “fs.s3a.performance.flags” to take list of performance flags
-
-S3A now supports configuration property `fs.s3a.performance.flag` for controlling activation of
-multiple performance optimizations. Refer to the S3A performance documentation for details.
+HADOOP-19256](https://issues.apache.org/jira/browse/HADOOP-19256) S3A: Adds support for S3 Conditional Writes.
ABFS improvements
-----------------
**Improvement**
-[HADOOP-18516](https://issues.apache.org/jira/browse/HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation
-
-ABFS now supports authentication via a fixed Shared Access Signature token. Refer to ABFS
-documentation of configuration property `fs.azure.sas.fixed.token` for details.
-
-[HADOOP-19089](https://issues.apache.org/jira/browse/HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path
-
-[HADOOP-18869](https://issues.apache.org/jira/browse/HADOOP-18869) previously implemented support for xattrs on the root path in the 3.4.0 release. Support for this has been removed in 3.4.1 to prevent the need for calling container APIs.
-
-[HADOOP-19178](https://issues.apache.org/jira/browse/HADOOP-19178) WASB Driver Deprecation and eventual removal
-
-This release announces deprecation of the WASB file system in favor of ABFS. Refer to ABFS
-documentation for additional guidance.
-
-**Bug**
-
-[HADOOP-18542](https://issues.apache.org/jira/browse/HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
+[HADOOP-19226](https://issues.apache.org/jira/browse/HADOOP-19226) ABFS: [FnsOverBlob] Implementing Azure Rest APIs on
+Blob Endpoint for AbfsBlobClient.
-It is no longer necessary to specify a tenant and client ID in configuration for MSI authentication
-when running in an Azure instance.
+[HADOOP-19474](https://issues.apache.org/jira/browse/HADOOP-19474) ABFS: [FnsOverBlob] Listing Optimizations to avoid
Review Comment:
https://issues.apache.org/jira/browse/HADOOP-19543 also can be added.
##########
hadoop-project/src/site/markdown/index.md.vm:
##########
@@ -23,75 +23,33 @@ Overview of Changes
Users are encouraged to read the full set of release notes.
This page provides an overview of the major changes.
-Bulk Delete API
-----------------------------------------
-
-[HADOOP-18679](https://issues.apache.org/jira/browse/HADOOP-18679) Bulk Delete API.
-
-This release provides an API to perform bulk delete of files/objects
-in an object store or filesystem.
-
New binary distribution
-----------------------
-[HADOOP-19083](https://issues.apache.org/jira/browse/HADOOP-19083) provide hadoop binary tarball without aws v2 sdk
-
-Hadoop has added a new variant of the binary distribution tarball, labeled with "lean" in the file
-name. This tarball excludes the full AWS SDK v2 bundle, resulting in approximately 50% reduction in
-file size.
+As of v3.4.2, Hadoop will only be distributed with a lean tarball, which excludes the full AWS SDK v2 bundle to reduce
+overall file size. This release has been tested with AWS SDK v2 2.29.52, which can be downloaded from Maven
+[here](https://mvnrepository.com/artifact/software.amazon.awssdk/aws-sdk-java/2.29.52).
S3A improvements
----------------
**Improvement**
-[HADOOP-18886](https://issues.apache.org/jira/browse/HADOOP-18886) S3A: AWS SDK V2 Migration: stabilization and S3Express
-
-This release completes stabilization efforts on the AWS SDK v2 migration and support of Amazon S3
-Express One Zone storage. S3 Select is no longer supported.
-
-[HADOOP-18993](https://issues.apache.org/jira/browse/HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)
-
-This introduces configuration property `fs.s3a.classloader.isolation`, which defaults to `true`.
-Set to `false` to disable S3A classloader isolation, which can be useful for installing custom
-credential providers in user-provided jars.
-
-[HADOOP-19047](https://issues.apache.org/jira/browse/HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits
-
-The S3A magic committer now supports configuration property
-`fs.s3a.committer.magic.track.commits.in.memory.enabled`. Set this to `true` to track commits in
-memory instead of on the file system, which reduces the number of remote calls.
+[HADOOP-19363](https://issues.apache.org/jira/browse/HADOOP-19363) S3A: Support analytics-accelerator-s3 input streams
+for parquet read performance.
-[HADOOP-19161](https://issues.apache.org/jira/browse/HADOOP-19161) S3A: option “fs.s3a.performance.flags” to take list of performance flags
-
-S3A now supports configuration property `fs.s3a.performance.flag` for controlling activation of
-multiple performance optimizations. Refer to the S3A performance documentation for details.
+HADOOP-19256](https://issues.apache.org/jira/browse/HADOOP-19256) S3A: Adds support for S3 Conditional Writes.
ABFS improvements
-----------------
**Improvement**
-[HADOOP-18516](https://issues.apache.org/jira/browse/HADOOP-18516) [ABFS]: Support fixed SAS token config in addition to Custom SASTokenProvider Implementation
-
-ABFS now supports authentication via a fixed Shared Access Signature token. Refer to ABFS
-documentation of configuration property `fs.azure.sas.fixed.token` for details.
-
-[HADOOP-19089](https://issues.apache.org/jira/browse/HADOOP-19089) [ABFS] Reverting Back Support of setXAttr() and getXAttr() on root path
-
-[HADOOP-18869](https://issues.apache.org/jira/browse/HADOOP-18869) previously implemented support for xattrs on the root path in the 3.4.0 release. Support for this has been removed in 3.4.1 to prevent the need for calling container APIs.
-
-[HADOOP-19178](https://issues.apache.org/jira/browse/HADOOP-19178) WASB Driver Deprecation and eventual removal
-
-This release announces deprecation of the WASB file system in favor of ABFS. Refer to ABFS
-documentation for additional guidance.
-
-**Bug**
-
-[HADOOP-18542](https://issues.apache.org/jira/browse/HADOOP-18542) Azure Token provider requires tenant and client IDs despite being optional
+[HADOOP-19226](https://issues.apache.org/jira/browse/HADOOP-19226) ABFS: [FnsOverBlob] Implementing Azure Rest APIs on
Review Comment:
For FNSOverBlob support,may be we can use parent JIRA here as it will track all the related work items.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "John6665 (via GitHub)" <gi...@apache.org>.
John6665 commented on PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#issuecomment-3204892829
Good job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "ahmarsuhail (via GitHub)" <gi...@apache.org>.
ahmarsuhail commented on PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#issuecomment-3200748949
@anujmodi2021 could you please review the ABFS changes, and let me know if there's anything else you want to highlight.
@steveloughran anything else in S3A we want to highlight?
I'll merge this in tomorrow and kick off the new build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#issuecomment-3205356855
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 0s | | Docker mode activated. |
| -1 :x: | patch | 0m 38s | | https://github.com/apache/hadoop/pull/7887 does not apply to branch-3.4.2. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/7887 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7887/2/console |
| versions | git=2.34.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#issuecomment-3201009350
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 7m 4s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
|||| _ branch-3.4.2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 22s | | branch-3.4.2 passed |
| +1 :green_heart: | mvnsite | 0m 17s | | branch-3.4.2 passed |
| +1 :green_heart: | shadedclient | 43m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 20s | | The patch does not generate ASF License warnings. |
| | | 71m 0s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7887/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7887 |
| Optional Tests | dupname asflicense mvnsite codespell detsecrets |
| uname | Linux 85cc1ac66933 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.4.2 / 62e2fc17c26865e8b191c7b34617af0ea7a5fa95 |
| Max. process+thread count | 717 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7887/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19657. Updates site landing page. [hadoop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on code in PR #7887:
URL: https://github.com/apache/hadoop/pull/7887#discussion_r2286735439
##########
hadoop-project/src/site/markdown/index.md.vm:
##########
@@ -23,75 +23,33 @@ Overview of Changes
Users are encouraged to read the full set of release notes.
This page provides an overview of the major changes.
-Bulk Delete API
-----------------------------------------
-
-[HADOOP-18679](https://issues.apache.org/jira/browse/HADOOP-18679) Bulk Delete API.
-
-This release provides an API to perform bulk delete of files/objects
-in an object store or filesystem.
-
New binary distribution
-----------------------
-[HADOOP-19083](https://issues.apache.org/jira/browse/HADOOP-19083) provide hadoop binary tarball without aws v2 sdk
-
-Hadoop has added a new variant of the binary distribution tarball, labeled with "lean" in the file
-name. This tarball excludes the full AWS SDK v2 bundle, resulting in approximately 50% reduction in
-file size.
+As of v3.4.2, Hadoop will only be distributed with a lean tarball, which excludes the full AWS SDK v2 bundle to reduce
+overall file size. This release has been tested with AWS SDK v2 2.29.52, which can be downloaded from Maven
+[here](https://mvnrepository.com/artifact/software.amazon.awssdk/aws-sdk-java/2.29.52).
S3A improvements
----------------
**Improvement**
-[HADOOP-18886](https://issues.apache.org/jira/browse/HADOOP-18886) S3A: AWS SDK V2 Migration: stabilization and S3Express
-
-This release completes stabilization efforts on the AWS SDK v2 migration and support of Amazon S3
-Express One Zone storage. S3 Select is no longer supported.
-
-[HADOOP-18993](https://issues.apache.org/jira/browse/HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)
-
-This introduces configuration property `fs.s3a.classloader.isolation`, which defaults to `true`.
-Set to `false` to disable S3A classloader isolation, which can be useful for installing custom
-credential providers in user-provided jars.
-
-[HADOOP-19047](https://issues.apache.org/jira/browse/HADOOP-19047) Support InMemory Tracking Of S3A Magic Commits
-
-The S3A magic committer now supports configuration property
-`fs.s3a.committer.magic.track.commits.in.memory.enabled`. Set this to `true` to track commits in
-memory instead of on the file system, which reduces the number of remote calls.
+[HADOOP-19363](https://issues.apache.org/jira/browse/HADOOP-19363) S3A: Support analytics-accelerator-s3 input streams
+for parquet read performance.
-[HADOOP-19161](https://issues.apache.org/jira/browse/HADOOP-19161) S3A: option “fs.s3a.performance.flags” to take list of performance flags
-
-S3A now supports configuration property `fs.s3a.performance.flag` for controlling activation of
-multiple performance optimizations. Refer to the S3A performance documentation for details.
+HADOOP-19256](https://issues.apache.org/jira/browse/HADOOP-19256) S3A: Adds support for S3 Conditional Writes.
Review Comment:
formatting error due to missing leading `[`. @ahmarsuhail
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org