You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "manika137 (via GitHub)" <gi...@apache.org> on 2026/04/07 05:36:27 UTC
[PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
manika137 opened a new pull request, #8410:
URL: https://github.com/apache/hadoop/pull/8410
### Description of PR
JIRA: https://issues.apache.org/jira/browse/HADOOP-19854
If fs.azure.account.hns.enabled is not configured, ABFS driver issues a getAcl call using the DFS endpoint to detect the account type (HNS or FNS)
For FNS Blob accounts with Soft Delete enabled, this behavior leads to failures because the DFS endpoint used does not support Soft Delete semantics.
Temporary fix is to explicitly set the configuration: fs.azure.account.hns.enabled so the redundant DFS-based detection is bypassed and the issue does not occur.
Raising PR for the code fix
### How was this patch tested?
Test suite was run. Adding the results in the comments below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#issuecomment-4197543794
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 20m 9s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 53m 39s | | trunk passed |
| +1 :green_heart: | compile | 0m 59s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 42s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 25s | [/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt) | hadoop-tools/hadoop-azure: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 27s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 159m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8410 |
| JIRA Issue | HADOOP-19854 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 683389bd161d 5.15.0-174-generic #184-Ubuntu SMP Fri Mar 13 18:41:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dff7687d2dfb67e8587de9efe870093c14cbefb5 |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/1/testReport/ |
| Max. process+thread count | 591 (vs. ulimit of 10000) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/1/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#issuecomment-4243232858
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 26s | | trunk passed |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 3s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 0m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 22s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 34m 56s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 26s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 21s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 17s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not generate ASF License warnings. |
| | | 135m 43s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8410 |
| JIRA Issue | HADOOP-19854 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 22832e4a505f 5.15.0-174-generic #184-Ubuntu SMP Fri Mar 13 18:41:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4ead3fe63fdc5674f097216387e31e44bb19504a |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/3/testReport/ |
| Max. process+thread count | 587 (vs. ulimit of 10000) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/3/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "anujmodi2021 (via GitHub)" <gi...@apache.org>.
anujmodi2021 merged PR #8410:
URL: https://github.com/apache/hadoop/pull/8410
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#issuecomment-4214249834
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 57m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 59s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 42s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 34m 16s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 27s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 20s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 56s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not generate ASF License warnings. |
| | | 143m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/2/artifact/out/Dockerfile |
| JIRA Issue | HADOOP-19854 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ac1dafaebb35 5.15.0-174-generic #184-Ubuntu SMP Fri Mar 13 18:41:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 25bd0dcfeac0dff144a2ccc19a902a021fcbcfd9 |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/2/testReport/ |
| Max. process+thread count | 622 (vs. ulimit of 10000) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8410/2/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "manika137 (via GitHub)" <gi...@apache.org>.
manika137 commented on PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#issuecomment-4196789386
## Test Results
============================================================
HNS-OAuth-DFS
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 858, Failures: 0, Errors: 0, Skipped: 172
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23
============================================================
HNS-SharedKey-DFS
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 3
[ERROR] Tests run: 862, Failures: 0, Errors: 0, Skipped: 118
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 10
============================================================
AppendBlob-HNS-OAuth-DFS
============================================================
[ERROR] Tests run: 229, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 858, Failures: 0, Errors: 0, Skipped: 183
[WARNING] Tests run: 136, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23
============================================================
NonHNS-SharedKey-Blob
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 708, Failures: 0, Errors: 0, Skipped: 154
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 11
============================================================
NonHNS-OAuth-Blob
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 705, Failures: 0, Errors: 0, Skipped: 155
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24
============================================================
AppendBlob-NonHNS-OAuth-Blob
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 700, Failures: 0, Errors: 0, Skipped: 176
[WARNING] Tests run: 136, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24
============================================================
HNS-Oauth-DFS-IngressBlob
============================================================
[WARNING] Tests run: 228, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 732, Failures: 0, Errors: 0, Skipped: 181
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "anujmodi2021 (via GitHub)" <gi...@apache.org>.
anujmodi2021 commented on code in PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#discussion_r3055444700
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:
##########
@@ -422,17 +423,33 @@ private synchronized boolean getNamespaceEnabledInformationFromServer(
} catch (AbfsRestOperationException ex) {
// Get ACL status is a HEAD request, its response doesn't contain errorCode
// So can only rely on its status code to determine account type.
- if (HttpURLConnection.HTTP_BAD_REQUEST != ex.getStatusCode()) {
+ int status = ex.getStatusCode();
+ String message = ex.getMessage();
+
+ // Case 1: 409: Soft delete not supported
+ if (status == HttpURLConnection.HTTP_CONFLICT &&
+ message != null &&
+ message.contains(ERR_SOFT_DELETE_NOT_SUPPORTED)) {
+ /*
+ * HTTP_CONFLICT with soft delete not supported error indicates a FNS account.
+ * This occurs when:
+ * 1. FNS-Blob endpoint is used (used with DFS endpoint for getAcl call)
+ * 2. FNS-DFS endpoint is used (later internally converted to Blob)
+ * For both cases, namespace should be disabled. No need to throw exception.
Review Comment:
Also add a line that for both cases ultimately Blob endpoint will be used so this error is irrelevant as it comes only on DFS Endpoint
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:
##########
@@ -422,17 +423,33 @@ private synchronized boolean getNamespaceEnabledInformationFromServer(
} catch (AbfsRestOperationException ex) {
// Get ACL status is a HEAD request, its response doesn't contain errorCode
// So can only rely on its status code to determine account type.
- if (HttpURLConnection.HTTP_BAD_REQUEST != ex.getStatusCode()) {
+ int status = ex.getStatusCode();
+ String message = ex.getMessage();
+
+ // Case 1: 409: Soft delete not supported
+ if (status == HttpURLConnection.HTTP_CONFLICT &&
+ message != null &&
+ message.contains(ERR_SOFT_DELETE_NOT_SUPPORTED)) {
+ /*
+ * HTTP_CONFLICT with soft delete not supported error indicates a FNS account.
+ * This occurs when:
+ * 1. FNS-Blob endpoint is used (used with DFS endpoint for getAcl call)
Review Comment:
nit: used coming twice
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestGetNameSpaceEnabled.java:
##########
@@ -274,8 +278,11 @@ private void ensureGetAclDetermineHnsStatusAccuratelyInternal(int statusCode,
AbfsClient mockClient = mock(AbfsClient.class);
store.getAbfsConfiguration().setIsNamespaceEnabledAccountForTesting(Trilean.UNKNOWN);
doReturn(mockClient).when(store).getClient(AbfsServiceType.DFS);
+ String errorMsg = statusCode == HTTP_CONFLICT
+ ? ERR_SOFT_DELETE_NOT_SUPPORTED
+ : Integer.toString(statusCode);
AbfsRestOperationException ex = new AbfsRestOperationException(
- statusCode, null, Integer.toString(statusCode), null);
+ statusCode, null, errorMsg, null);
doThrow(ex).when(mockClient).getAclStatus(anyString(), any(TracingContext.class));
if (isExceptionExpected) {
Review Comment:
Need some more testing no just mock.
I understand that we can't control account's soft-delete feature but let's run all the integration tests on an account where soft-delete is enabled if not already done.
And in integration tests we can assert that if error code returned by server can either be Bad request of htp conflict.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "manika137 (via GitHub)" <gi...@apache.org>.
manika137 commented on code in PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#discussion_r3056014459
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:
##########
@@ -422,17 +423,33 @@ private synchronized boolean getNamespaceEnabledInformationFromServer(
} catch (AbfsRestOperationException ex) {
// Get ACL status is a HEAD request, its response doesn't contain errorCode
// So can only rely on its status code to determine account type.
- if (HttpURLConnection.HTTP_BAD_REQUEST != ex.getStatusCode()) {
+ int status = ex.getStatusCode();
+ String message = ex.getMessage();
+
+ // Case 1: 409: Soft delete not supported
+ if (status == HttpURLConnection.HTTP_CONFLICT &&
+ message != null &&
+ message.contains(ERR_SOFT_DELETE_NOT_SUPPORTED)) {
+ /*
+ * HTTP_CONFLICT with soft delete not supported error indicates a FNS account.
+ * This occurs when:
+ * 1. FNS-Blob endpoint is used (used with DFS endpoint for getAcl call)
+ * 2. FNS-DFS endpoint is used (later internally converted to Blob)
+ * For both cases, namespace should be disabled. No need to throw exception.
Review Comment:
Taken
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19854: ABFS. Namespace Detection for FNS Accounts with Soft Delete Enabled [hadoop]
Posted by "manika137 (via GitHub)" <gi...@apache.org>.
manika137 commented on code in PR #8410:
URL: https://github.com/apache/hadoop/pull/8410#discussion_r3055904413
##########
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestGetNameSpaceEnabled.java:
##########
@@ -274,8 +278,11 @@ private void ensureGetAclDetermineHnsStatusAccuratelyInternal(int statusCode,
AbfsClient mockClient = mock(AbfsClient.class);
store.getAbfsConfiguration().setIsNamespaceEnabledAccountForTesting(Trilean.UNKNOWN);
doReturn(mockClient).when(store).getClient(AbfsServiceType.DFS);
+ String errorMsg = statusCode == HTTP_CONFLICT
+ ? ERR_SOFT_DELETE_NOT_SUPPORTED
+ : Integer.toString(statusCode);
AbfsRestOperationException ex = new AbfsRestOperationException(
- statusCode, null, Integer.toString(statusCode), null);
+ statusCode, null, errorMsg, null);
doThrow(ex).when(mockClient).getAclStatus(anyString(), any(TracingContext.class));
if (isExceptionExpected) {
Review Comment:
Sure, will add integration tests
I have run the test suite with account's soft-delete enabled as well. Apart from the list failing (with startfrom param) that we have a fix coming up for- we're good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org