You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "sadanand48 (via GitHub)" <gi...@apache.org> on 2026/09/16 08:12:37 UTC
[PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
sadanand48 opened a new pull request, #8741:
URL: https://github.com/apache/hadoop/pull/8741
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
When HDFS Trash is enabled, rm is executed as an internal rename to .Trash using Options.Rename.TO_TRASH.
Today, external authorization plugins (for example Ranger via checkPermissionWithContext) receive operationName=rename, but do not get an explicit trash-intent signal. As a result, rm (to trash) and mv appear identical from authorization/audit context.
This JIRA proposes adding and propagating a dedicated context flag (for example renameToTrash) in INodeAttributeProvider.AuthorizationContext so external enforcers can distinguish:
rm with trash enabled -> operationName=rename, renameToTrash=true
mv -> operationName=rename, renameToTrash=false
rm skipTrash > operationName=delete
Scope (HDFS side)
Add renameToTrash field to AuthorizationContext and its builder/accessors.
Propagate the flag through FSPermissionChecker when building context for checkPermissionWithContext.
In FSNamesystem.renameTo(..., Options.Rename... options), set the flag based on presence of Options.Rename.TO_TRASH, and clear it in finally to avoid ThreadLocal leakage.
### How was this patch tested?
Unit tests
### For code changes:
- [ ] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the endpoint
declared according to the connector-specific documentation? *Note: Automated CI
testing doesn't cover all cases so manual testing with cloud storage is still
required.*
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
### AI Tooling
If an AI tool was used:
- [ ] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [ ] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#discussion_r4048725296
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java:
##########
@@ -163,6 +165,14 @@ public static void setOperationType(String opType) {
operationType.set(opType);
}
+ public static void setRenameToTrash(boolean value) {
Review Comment:
done.
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3387,6 +3388,8 @@ boolean renameTo(String src, String dst, boolean logRetryCache)
} catch (AccessControlException e) {
logAuditEvent(false, operationName, src, dst, null);
throw e;
+ } finally {
+ FSPermissionChecker.setRenameToTrash(false);
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#issuecomment-5697590977
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 7m 28s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 27m 28s | | trunk passed |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | compile | 1m 4s | | trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 2m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 14s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 47s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 46s | | the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 0m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 44s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 141 unchanged - 0 fixed = 142 total (was 141) |
| +1 :green_heart: | mvnsite | 0m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 2m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 58s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 183m 33s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not generate ASF License warnings. |
| | | 268m 9s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8741 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 06e04040eadc 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6855a355af51482e2ebfafda583ae73db2e96d62 |
| Default Java | Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12+8-1-24.04-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/testReport/ |
| Max. process+thread count | 4074 (vs. ulimit of 10000) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/1/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #8741:
URL: https://github.com/apache/hadoop/pull/8741
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "Hexiaoqiao (via GitHub)" <gi...@apache.org>.
Hexiaoqiao commented on PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#issuecomment-5725171263
> Today, external authorization plugins (for example Ranger via checkPermissionWithContext) receive operationName=rename, but do not get an explicit trash-intent signal. As a result, rm (to trash) and mv appear identical from authorization/audit context.
Sorry, didn't get this point. For the external authorization plugins, is it enough to use operation name and src/dest path? How will Ranger use this rename options. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#issuecomment-5726118816
> Sorry, didn't get this point. For the external authorization plugins, is it enough to use operation name and src/dest path? How will Ranger use this rename options. Thanks.
This is mainly for compliance tracking in Ranger Audits as when trash is enabled rm and mv operations look identical .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#issuecomment-5736261129
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 23s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 28m 4s | | trunk passed |
| +1 :green_heart: | compile | 0m 59s | | trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | compile | 1m 2s | | trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 2m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 22s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 44s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 141 unchanged - 0 fixed = 142 total (was 141) |
| +1 :green_heart: | mvnsite | 0m 51s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK Ubuntu-21.0.12+8-1-24.04-Ubuntu |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| +1 :green_heart: | spotbugs | 2m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 191m 37s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not generate ASF License warnings. |
| | | 269m 18s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeLifeline |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8741 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cd68d6bfc6bc 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 65649ae5187526397bac7f44db27fc23d85c6617 |
| Default Java | Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12+8-1-24.04-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20+8-1-24.04-Ubuntu |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/testReport/ |
| Max. process+thread count | 4114 (vs. ulimit of 10000) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8741/2/console |
| versions | git=2.43.0 maven=3.9.15 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HDFS-17980. Propagate Rename.TO_TRASH in authorization context for external enforcers. [hadoop]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #8741:
URL: https://github.com/apache/hadoop/pull/8741#discussion_r4041572048
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:
##########
@@ -3387,6 +3388,8 @@ boolean renameTo(String src, String dst, boolean logRetryCache)
} catch (AccessControlException e) {
logAuditEvent(false, operationName, src, dst, null);
throw e;
+ } finally {
+ FSPermissionChecker.setRenameToTrash(false);
Review Comment:
this one's redundant.
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java:
##########
@@ -163,6 +165,14 @@ public static void setOperationType(String opType) {
operationType.set(opType);
}
+ public static void setRenameToTrash(boolean value) {
Review Comment:
please add javadoc to explain what this flag mean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org