You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/10/24 17:35:41 UTC
[GitHub] [hadoop] xinglin opened a new pull request, #5071: Hdfs 16816
xinglin opened a new pull request, #5071:
URL: https://github.com/apache/hadoop/pull/5071
### Description of PR
In RBF, trash files are moved to trash root under user's home dir at the corresponding namespace/namenode where the files reside. This was added in [HDFS-16024](https://issues.apache.org/jira/browse/HDFS-16024). When the user home dir is not created before-hand at a namenode, we run into permission denied exceptions when trying to create the parent dir for the trash file before moving the file into it. We propose to enhance Router, to auto-create a user home's dir at the namenode for trash paths, using router's identity (which is assumed to be a super-user).
### How was this patch tested?
Add a new unit test (`testMoveToTrashAutoCreateUserHome`) to TestRouterTrash.
mvn test -Dtest="TestRouterTrash"
```
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hdfs.server.federation.router.TestRouterTrash
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.326 s - in org.apache.hadoop.hdfs.server.federation.router.TestRouterTrash
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
```
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1291288174
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 35s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 32s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 19s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/4/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 25s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 2s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 35m 43s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not generate ASF License warnings. |
| | | 142m 34s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c063aa131c1b 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8563b455f6c32f53fa683f43638fdda20e169846 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/4/testReport/ |
| Max. process+thread count | 3174 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1290933389
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 0s | | Docker mode activated. |
| -1 :x: | docker | 4m 23s | | Docker failed to build run-specific yetus/hadoop:tp-29905}. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/3/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1290017891
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 22s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 59s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| -1 :x: | javac | 0m 40s | [/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt) | hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 55 unchanged - 0 fixed = 56 total (was 55) |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 34s | [/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 1 new + 55 unchanged - 0 fixed = 56 total (was 55) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 20s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4) |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 5s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 35m 32s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not generate ASF License warnings. |
| | | 157m 52s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 071cfe371624 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1dae24f486414359c7cf280c59f21c8b2133196b |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/testReport/ |
| Max. process+thread count | 3177 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003606901
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterTrash.java:
##########
@@ -144,6 +144,40 @@ private boolean addMountTable(final MountTable entry) throws IOException {
return addResponse.getStatus();
}
+ @Test
+ public void testMoveToTrashAutoCreateUserHome() throws IOException, URISyntaxException, InterruptedException {
Review Comment:
CAn you create a negative test case: where the user directory already exists we don't create it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006010839
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterTrash.java:
##########
@@ -144,6 +145,95 @@ private boolean addMountTable(final MountTable entry) throws IOException {
return addResponse.getStatus();
}
+ /**
+ * We manually create the trash root for TEST_USER. Then, moveToTrash
+ * shall not cause the router to create user's home dir anymore.
+ *
+ * We compare the modification of user home dir before and after moveToTrash,
+ * to verify that we don't try to re-create user home dir once it already
+ * exists. We need to pre-create the trash root, instead of user home dir,
+ * because adding a subdir to a dir will change the dir's modification time.
+ */
+ @Test
+ public void testMoveToTrashAutoCreateUserHomeAlreadyExisted()
+ throws IOException, URISyntaxException, InterruptedException {
+ MountTable addEntry = MountTable.newInstance(MOUNT_POINT,
+ Collections.singletonMap(ns0, MOUNT_POINT));
+ assertTrue(addMountTable(addEntry));
+ String testUserHome = "/user/" + TEST_USER;
+ String testUserTrashRoot = testUserHome + "/.Trash";
+
+ // Set owner to TEST_USER for root dir
+ DFSClient superUserClient = nnContext.getClient();
+ superUserClient.setOwner("/", TEST_USER, TEST_USER);
+
+ // Create MOUNT_POINT and user's trash root
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(TEST_USER);
+ DFSClient client = nnContext.getClient(ugi);
+ client.mkdirs(MOUNT_POINT, new FsPermission("777"), true);
+ assertTrue(client.exists(MOUNT_POINT));
+ client.mkdirs(testUserTrashRoot, new FsPermission("775"), true);
+ assertTrue(client.exists(testUserTrashRoot));
+
+ // Create test file
+ client.create(FILE, true);
+ Path filePath = new Path(FILE);
+
+ // Get the fileStatus before moveToTrash
+ HdfsFileStatus status = client.getFileInfo(testUserHome);
+
+ // Test moveToTrash by TEST_USER
+ String trashPath = "/user/" + TEST_USER + "/.Trash/Current" + FILE;
+ Configuration routerConf = routerContext.getConf();
+ FileSystem fs = DFSTestUtil.getFileSystemAs(ugi, routerConf);
+ Trash trash = new Trash(fs, routerConf);
+ assertTrue(trash.moveToTrash(filePath));
+ assertTrue(nnFs.exists(new Path(trashPath)));
+
+ HdfsFileStatus status2 = client.getFileInfo(testUserHome);
+ assertEquals(status.getModificationTime(), status2.getModificationTime());
Review Comment:
Does modtime change if the directory already exists? i.e. we do an "if not exists create"
This can be unit tested with mocks where we mock out the rpc client to observe what is actually called to avoid this ambiguity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006258981
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -734,6 +806,11 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent)
new Class<?>[] {String.class, FsPermission.class, boolean.class},
new RemoteParam(), masked, createParent);
+ // Auto-create user home dir for a trash path.
+ // moveToTrash() will first call fs.mkdirs() to create the parent dir, before calling rename()
+ // to move the file into it. As a result, we need to create user home dir in mkdirs().
+ createUserHomeForTrashPath(src, locations);
Review Comment:
Should this be a flag so we enable it via config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1291473368
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 16s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 5s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 35m 23s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 141m 10s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 174626bea195 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1de60c62ba7199d53f603eb10f7e33cb4fb72cf7 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/6/testReport/ |
| Max. process+thread count | 3175 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1294021067
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 52s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 12s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 30s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 36m 21s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 144m 6s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/12/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux b556edd6e5e5 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a3c07dac9b460459a0edfc0803b976b652a7317e |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/12/testReport/ |
| Max. process+thread count | 3465 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006314263
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java:
##########
@@ -997,6 +1008,20 @@ public <T> T invokeSequential(
expectedResultValue).getResult();
}
+ /**
+ * Same as the above invokeSequential, but with router ugi.
+ */
+ public <T> T invokeSequentialAsRouter(
Review Comment:
don't want to copy the whole same section from invokeSequential. We will keep it short here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006260919
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -734,6 +806,11 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent)
new Class<?>[] {String.class, FsPermission.class, boolean.class},
new RemoteParam(), masked, createParent);
+ // Auto-create user home dir for a trash path.
+ // moveToTrash() will first call fs.mkdirs() to create the parent dir, before calling rename()
+ // to move the file into it. As a result, we need to create user home dir in mkdirs().
+ createUserHomeForTrashPath(src, locations);
Review Comment:
I am open to that option. If we control with a flag, should this be turned on by default or should it be off?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003591256
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,49 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ // Create missing user home dirs for trash paths.
+ // We assume the router is running with super-user privilege (can create
+ // user home dir in /user dir).
+ private void createUserHomeForTrashPath(List<RemoteLocation> locations) throws IOException {
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Identify missing trash roots
+ for(RemoteLocation loc: locations) {
+
+ String path = loc.getDest();
+ // Continue if not a trash path
+ if (!MountTableResolver.isTrashPath(path)) {
+ continue;
+ }
+
+ // Check whether user home dir exists at the destination namespace
+ String trashRoot = MountTableResolver.getTrashRoot();
+ String userHome = new Path(trashRoot).getParent().toUri().getPath();
+ RemoteLocation userHomeLoc = new RemoteLocation(loc, userHome);
+ RemoteMethod method = new RemoteMethod("getFileInfo", new Class<?>[] {String.class}, new RemoteParam());
+ HdfsFileStatus ret = rpcClient.invokeSingle(userHomeLoc, method, HdfsFileStatus.class);
+ if (ret == null) {
+ missingUserHomes.add(userHomeLoc);
+ }
+ }
+
+ if (!missingUserHomes.isEmpty()) {
Review Comment:
Is this logic correct? When missingUserHomes is populated once while looping through locations this will always be true.
Should this instead be in a separate loop over missingUserHomes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1293090033
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 56s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 10s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 5s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 35m 20s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not generate ASF License warnings. |
| | | 139m 55s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 679fe1c37824 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 74624770b0e1d2a8c69087a09d52671ebf975da8 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/10/testReport/ |
| Max. process+thread count | 3110 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1293989334
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 7s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 52s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 40s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 41s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 25m 22s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not generate ASF License warnings. |
| | | 124m 14s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 675d4c814a6d 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d8adefffaf7b57d9135109561b1673376bdd536d |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/11/testReport/ |
| Max. process+thread count | 3823 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006014094
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,78 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ /**
+ * Create missing user home dirs for trash paths.
+ *
+ * When there are multiple possible destination locations for a trash path,
+ * we create user home dirs at all destination locations.
+ *
+ * We assume the router is running with super-user privilege (can create
+ * user home dir in /user dir).
+ *
+ * @param src source location
+ * @param locations destination locations
+ */
+ private void createUserHomeForTrashPath(String src,
+ List<RemoteLocation> locations)
+ throws IOException {
+ List<RemoteLocation> userHomes = new ArrayList<>();
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Get home dirs for trash paths
+ for (RemoteLocation loc : locations) {
+
+ String path = loc.getDest();
+ // Continue if not a trash path
+ if (!MountTableResolver.isTrashPath(path)) {
+ continue;
+ }
+
+ String trashRoot = MountTableResolver.getTrashRoot();
+ String userHome = new Path(trashRoot).getParent().toUri().getPath();
+ RemoteLocation userHomeLoc = new RemoteLocation(loc, userHome);
+ userHomes.add(userHomeLoc);
+ }
+
+ // Use getFileInfo to identify missing user homes
+ for (RemoteLocation userHomeLoc : userHomes) {
+ RemoteMethod method =
+ new RemoteMethod("getFileInfo", new Class<?>[]{String.class},
+ new RemoteParam());
+ HdfsFileStatus ret =
+ rpcClient.invokeSingle(userHomeLoc, method, HdfsFileStatus.class);
+ if (ret == null) {
+ missingUserHomes.add(userHomeLoc);
+ }
+ }
+
+ // Create missing user home dirs and set owner
+ for(RemoteLocation loc: missingUserHomes) {
Review Comment:
My concern is this function is doing too much; it first creates / filters a list (involves IO) and then creates directories (Also IO). The logic that involces IO can be encapsulated with a lambda so we can test the logic of the function in-isolation of the result of the IO. Alternatively we can mock the RPC client to observe what is actually called by this function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] virajith commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
virajith commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1011042738
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -734,6 +811,13 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent)
new Class<?>[] {String.class, FsPermission.class, boolean.class},
new RemoteParam(), masked, createParent);
+ // Auto-create user home dir for a trash path.
+ // moveToTrash() will first call fs.mkdirs() to create the parent dir, before calling rename()
+ // to move the file into it. As a result, we need to create user home dir in mkdirs().
+ if (autoCreateUserHomeForTrash) {
+ createUserHomeForTrashPath(src, locations);
Review Comment:
I don't think you should be piggybacking on {{mkdirs}} to get this created. If this is a useful feature to have, I'd suggest adding a new FileSystem API named something like {{provisionTrashPath()}} similar to {{provisionEZTrash}} and call it in {{TrashPolicy#moveToTrash}} before {{mkdirs}}.
Can you also call out why you need this vs. provisioning user trash paths as part of the (off band) process of provisioning the user home directories on HDFS?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006017718
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterTrash.java:
##########
@@ -144,6 +145,95 @@ private boolean addMountTable(final MountTable entry) throws IOException {
return addResponse.getStatus();
}
+ /**
+ * We manually create the trash root for TEST_USER. Then, moveToTrash
+ * shall not cause the router to create user's home dir anymore.
+ *
+ * We compare the modification of user home dir before and after moveToTrash,
+ * to verify that we don't try to re-create user home dir once it already
+ * exists. We need to pre-create the trash root, instead of user home dir,
+ * because adding a subdir to a dir will change the dir's modification time.
+ */
+ @Test
+ public void testMoveToTrashAutoCreateUserHomeAlreadyExisted()
+ throws IOException, URISyntaxException, InterruptedException {
+ MountTable addEntry = MountTable.newInstance(MOUNT_POINT,
+ Collections.singletonMap(ns0, MOUNT_POINT));
+ assertTrue(addMountTable(addEntry));
+ String testUserHome = "/user/" + TEST_USER;
+ String testUserTrashRoot = testUserHome + "/.Trash";
+
+ // Set owner to TEST_USER for root dir
+ DFSClient superUserClient = nnContext.getClient();
+ superUserClient.setOwner("/", TEST_USER, TEST_USER);
+
+ // Create MOUNT_POINT and user's trash root
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(TEST_USER);
+ DFSClient client = nnContext.getClient(ugi);
+ client.mkdirs(MOUNT_POINT, new FsPermission("777"), true);
+ assertTrue(client.exists(MOUNT_POINT));
+ client.mkdirs(testUserTrashRoot, new FsPermission("775"), true);
+ assertTrue(client.exists(testUserTrashRoot));
+
+ // Create test file
+ client.create(FILE, true);
+ Path filePath = new Path(FILE);
+
+ // Get the fileStatus before moveToTrash
+ HdfsFileStatus status = client.getFileInfo(testUserHome);
+
+ // Test moveToTrash by TEST_USER
+ String trashPath = "/user/" + TEST_USER + "/.Trash/Current" + FILE;
+ Configuration routerConf = routerContext.getConf();
+ FileSystem fs = DFSTestUtil.getFileSystemAs(ugi, routerConf);
+ Trash trash = new Trash(fs, routerConf);
+ assertTrue(trash.moveToTrash(filePath));
+ assertTrue(nnFs.exists(new Path(trashPath)));
+
+ HdfsFileStatus status2 = client.getFileInfo(testUserHome);
+ assertEquals(status.getModificationTime(), status2.getModificationTime());
Review Comment:
modtime won't change when the dir already exists and its content does not get modified (add a subdir or remove a subdir).
We don't mock rpc client in TestRouterTrash. If we use mocked RPC client for testing, we should probably put it in a separate new class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006245564
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterTrash.java:
##########
@@ -144,6 +145,95 @@ private boolean addMountTable(final MountTable entry) throws IOException {
return addResponse.getStatus();
}
+ /**
+ * We manually create the trash root for TEST_USER. Then, moveToTrash
+ * shall not cause the router to create user's home dir anymore.
+ *
+ * We compare the modification of user home dir before and after moveToTrash,
+ * to verify that we don't try to re-create user home dir once it already
+ * exists. We need to pre-create the trash root, instead of user home dir,
+ * because adding a subdir to a dir will change the dir's modification time.
+ */
+ @Test
+ public void testMoveToTrashAutoCreateUserHomeAlreadyExisted()
+ throws IOException, URISyntaxException, InterruptedException {
+ MountTable addEntry = MountTable.newInstance(MOUNT_POINT,
+ Collections.singletonMap(ns0, MOUNT_POINT));
+ assertTrue(addMountTable(addEntry));
+ String testUserHome = "/user/" + TEST_USER;
+ String testUserTrashRoot = testUserHome + "/.Trash";
+
+ // Set owner to TEST_USER for root dir
+ DFSClient superUserClient = nnContext.getClient();
+ superUserClient.setOwner("/", TEST_USER, TEST_USER);
+
+ // Create MOUNT_POINT and user's trash root
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(TEST_USER);
+ DFSClient client = nnContext.getClient(ugi);
+ client.mkdirs(MOUNT_POINT, new FsPermission("777"), true);
+ assertTrue(client.exists(MOUNT_POINT));
+ client.mkdirs(testUserTrashRoot, new FsPermission("775"), true);
+ assertTrue(client.exists(testUserTrashRoot));
+
+ // Create test file
+ client.create(FILE, true);
+ Path filePath = new Path(FILE);
+
+ // Get the fileStatus before moveToTrash
+ HdfsFileStatus status = client.getFileInfo(testUserHome);
+
+ // Test moveToTrash by TEST_USER
+ String trashPath = "/user/" + TEST_USER + "/.Trash/Current" + FILE;
+ Configuration routerConf = routerContext.getConf();
+ FileSystem fs = DFSTestUtil.getFileSystemAs(ugi, routerConf);
+ Trash trash = new Trash(fs, routerConf);
+ assertTrue(trash.moveToTrash(filePath));
+ assertTrue(nnFs.exists(new Path(trashPath)));
+
+ HdfsFileStatus status2 = client.getFileInfo(testUserHome);
+ assertEquals(status.getModificationTime(), status2.getModificationTime());
Review Comment:
replaced with a test using a mocked object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006261114
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java:
##########
@@ -997,6 +1008,20 @@ public <T> T invokeSequential(
expectedResultValue).getResult();
}
+ /**
+ * Same as the above invokeSequential, but with router ugi.
+ */
+ public <T> T invokeSequentialAsRouter(
Review Comment:
Nit Javadoc on public function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1291348446
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 17m 17s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 48s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 41s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 27s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 29s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/5/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 38s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 58s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 38m 8s | | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not generate ASF License warnings. |
| | | 153m 16s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux dae824abd7c5 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2503f7bb54236e0ac37db93d942c27d5261b396e |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/5/testReport/ |
| Max. process+thread count | 3407 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006313820
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -734,6 +806,11 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent)
new Class<?>[] {String.class, FsPermission.class, boolean.class},
new RemoteParam(), masked, createParent);
+ // Auto-create user home dir for a trash path.
+ // moveToTrash() will first call fs.mkdirs() to create the parent dir, before calling rename()
+ // to move the file into it. As a result, we need to create user home dir in mkdirs().
+ createUserHomeForTrashPath(src, locations);
Review Comment:
added a config to control it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003631842
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,49 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ // Create missing user home dirs for trash paths.
+ // We assume the router is running with super-user privilege (can create
+ // user home dir in /user dir).
+ private void createUserHomeForTrashPath(List<RemoteLocation> locations) throws IOException {
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Identify missing trash roots
+ for(RemoteLocation loc: locations) {
Review Comment:
Sure, I will split the first for loop into two for loops. The third is already taken care of by invokeSequentialInternal().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006029768
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,78 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ /**
+ * Create missing user home dirs for trash paths.
+ *
+ * When there are multiple possible destination locations for a trash path,
+ * we create user home dirs at all destination locations.
+ *
+ * We assume the router is running with super-user privilege (can create
+ * user home dir in /user dir).
+ *
+ * @param src source location
+ * @param locations destination locations
+ */
+ private void createUserHomeForTrashPath(String src,
+ List<RemoteLocation> locations)
+ throws IOException {
+ List<RemoteLocation> userHomes = new ArrayList<>();
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Get home dirs for trash paths
+ for (RemoteLocation loc : locations) {
+
+ String path = loc.getDest();
+ // Continue if not a trash path
+ if (!MountTableResolver.isTrashPath(path)) {
+ continue;
+ }
+
+ String trashRoot = MountTableResolver.getTrashRoot();
+ String userHome = new Path(trashRoot).getParent().toUri().getPath();
+ RemoteLocation userHomeLoc = new RemoteLocation(loc, userHome);
+ userHomes.add(userHomeLoc);
+ }
+
+ // Use getFileInfo to identify missing user homes
+ for (RemoteLocation userHomeLoc : userHomes) {
+ RemoteMethod method =
+ new RemoteMethod("getFileInfo", new Class<?>[]{String.class},
+ new RemoteParam());
+ HdfsFileStatus ret =
+ rpcClient.invokeSingle(userHomeLoc, method, HdfsFileStatus.class);
+ if (ret == null) {
+ missingUserHomes.add(userHomeLoc);
+ }
+ }
+
+ // Create missing user home dirs and set owner
+ for(RemoteLocation loc: missingUserHomes) {
Review Comment:
I don't think this function is too complicated. It just does two things: first check whether a home dir exists and then create these home dirs which don't exist yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
goiri commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1012250183
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -734,6 +811,13 @@ public boolean mkdirs(String src, FsPermission masked, boolean createParent)
new Class<?>[] {String.class, FsPermission.class, boolean.class},
new RemoteParam(), masked, createParent);
+ // Auto-create user home dir for a trash path.
+ // moveToTrash() will first call fs.mkdirs() to create the parent dir, before calling rename()
+ // to move the file into it. As a result, we need to create user home dir in mkdirs().
+ if (autoCreateUserHomeForTrash) {
+ createUserHomeForTrashPath(src, locations);
Review Comment:
Given that we already have provisionEZTrash(), it does make sense to do the same for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by "xinglin (via GitHub)" <gi...@apache.org>.
xinglin commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1585083991
this PR is no longer pursued.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003605800
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,49 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ // Create missing user home dirs for trash paths.
+ // We assume the router is running with super-user privilege (can create
+ // user home dir in /user dir).
+ private void createUserHomeForTrashPath(List<RemoteLocation> locations) throws IOException {
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Identify missing trash roots
+ for(RemoteLocation loc: locations) {
+
+ String path = loc.getDest();
+ // Continue if not a trash path
+ if (!MountTableResolver.isTrashPath(path)) {
+ continue;
+ }
+
+ // Check whether user home dir exists at the destination namespace
+ String trashRoot = MountTableResolver.getTrashRoot();
+ String userHome = new Path(trashRoot).getParent().toUri().getPath();
+ RemoteLocation userHomeLoc = new RemoteLocation(loc, userHome);
+ RemoteMethod method = new RemoteMethod("getFileInfo", new Class<?>[] {String.class}, new RemoteParam());
+ HdfsFileStatus ret = rpcClient.invokeSingle(userHomeLoc, method, HdfsFileStatus.class);
+ if (ret == null) {
+ missingUserHomes.add(userHomeLoc);
+ }
+ }
+
+ if (!missingUserHomes.isEmpty()) {
Review Comment:
Hi @mkuchenbecker,
When missingUserHomes is empty, it means all user home dirs have been created and thus we don't need to create user home dir anymore.
There is a for loop inside `invokeSequentialInternal`(), to loop over locations. `invokeSequentialAsRouter`() will call `invokeSequentialInternal`(). I used invokeSequential, since I believe a trash path will mostly like live at a single namenode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003605313
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,49 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ // Create missing user home dirs for trash paths.
+ // We assume the router is running with super-user privilege (can create
+ // user home dir in /user dir).
+ private void createUserHomeForTrashPath(List<RemoteLocation> locations) throws IOException {
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Identify missing trash roots
+ for(RemoteLocation loc: locations) {
Review Comment:
I would consider structuring this differently for readability / testability:
1. Map the original list of locations to a List of Home Directories. (i.e. given a list, make another list is easily testable)
2. Filter the list to the list of locations that don't exist. (i.e. given a list, make another list)
3. Create those directories that don't exist. (for each element in the list, call function X)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
xinglin commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1289991163
yetus may come back with unit test failure of `TestRouterRPCMultipleDestinationMountTableResolver`. I have tested that this class is failing on trunk non-deterministically as well. Created another ticket (HDFS-16818) to handle this unit test failure.
`TestRouterRPCMultipleDestinationMountTableResolver` mainly tests with the following paths, which shouldn't be impacted with the change in this PR.
```
Path mountDir = new Path("/mount/dir/dir");
Path nameSpaceFile = new Path("/tmp/dir/file");
Path mountFile = new Path("/mount/dir/file");
Path mountEntry = new Path("/mount");
Path mountDest = new Path("/tmp");
Path nameSpaceDir = new Path("/tmp/dir/dir");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006014094
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,78 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ /**
+ * Create missing user home dirs for trash paths.
+ *
+ * When there are multiple possible destination locations for a trash path,
+ * we create user home dirs at all destination locations.
+ *
+ * We assume the router is running with super-user privilege (can create
+ * user home dir in /user dir).
+ *
+ * @param src source location
+ * @param locations destination locations
+ */
+ private void createUserHomeForTrashPath(String src,
+ List<RemoteLocation> locations)
+ throws IOException {
+ List<RemoteLocation> userHomes = new ArrayList<>();
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Get home dirs for trash paths
+ for (RemoteLocation loc : locations) {
+
+ String path = loc.getDest();
+ // Continue if not a trash path
+ if (!MountTableResolver.isTrashPath(path)) {
+ continue;
+ }
+
+ String trashRoot = MountTableResolver.getTrashRoot();
+ String userHome = new Path(trashRoot).getParent().toUri().getPath();
+ RemoteLocation userHomeLoc = new RemoteLocation(loc, userHome);
+ userHomes.add(userHomeLoc);
+ }
+
+ // Use getFileInfo to identify missing user homes
+ for (RemoteLocation userHomeLoc : userHomes) {
+ RemoteMethod method =
+ new RemoteMethod("getFileInfo", new Class<?>[]{String.class},
+ new RemoteParam());
+ HdfsFileStatus ret =
+ rpcClient.invokeSingle(userHomeLoc, method, HdfsFileStatus.class);
+ if (ret == null) {
+ missingUserHomes.add(userHomeLoc);
+ }
+ }
+
+ // Create missing user home dirs and set owner
+ for(RemoteLocation loc: missingUserHomes) {
Review Comment:
My concern is this function is doing too much and is hard to test; it first creates / filters a list (involves IO) and then creates directories (Also IO). The logic that involces IO can be encapsulated with a lambda so we can test the logic of the function in-isolation of the result of the IO. Alternatively we can mock the RPC client to observe what is actually called by this function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] xinglin closed pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by "xinglin (via GitHub)" <gi...@apache.org>.
xinglin closed pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
URL: https://github.com/apache/hadoop/pull/5071
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1289525173
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 22s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| -1 :x: | javac | 0m 40s | [/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt) | hadoop-hdfs-project_hadoop-hdfs-rbf-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 1 new + 55 unchanged - 0 fixed = 56 total (was 55) |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 34s | [/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/artifact/out/results-compile-javac-hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt) | hadoop-hdfs-project_hadoop-hdfs-rbf-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 1 new + 55 unchanged - 0 fixed = 56 total (was 55) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 0m 23s | [/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 7 new + 4 unchanged - 0 fixed = 11 total (was 4) |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 3s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 38m 34s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not generate ASF License warnings. |
| | | 136m 49s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f91dcc080b25 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ec0ab34a520a2ca9e1e7fb76fc565eb4cfb6f75d |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/testReport/ |
| Max. process+thread count | 3429 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1003605313
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -723,6 +723,49 @@ public boolean delete(String src, boolean recursive) throws IOException {
}
}
+ // Create missing user home dirs for trash paths.
+ // We assume the router is running with super-user privilege (can create
+ // user home dir in /user dir).
+ private void createUserHomeForTrashPath(List<RemoteLocation> locations) throws IOException {
+ List<RemoteLocation> missingUserHomes = new ArrayList<>();
+
+ // Identify missing trash roots
+ for(RemoteLocation loc: locations) {
Review Comment:
I would consider structuring this differently for readability / testability:
1. Map the original list of locations to a List of Home Directories. (i.e. given a list, make another list is easily testable)
2. Filter the list to the list of locations that don't exist. (i.e. given a list)
3. Create those directories that don't exist. (for each element in the list, call function X)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
mkuchenbecker commented on code in PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#discussion_r1006260908
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java:
##########
@@ -946,6 +946,17 @@ public <T> T invokeSingle(final RemoteLocationContext location,
return ret;
}
+ // Same as invokeSingle but uses router's login identity
Review Comment:
Nit. Javadoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1292830019
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 34s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 21s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 53s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 35m 56s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 144m 25s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRPCMultipleDestinationMountTableResolver |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 66e7ee1dc096 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d73bf1396667f4b0b48274165ff5a6933a8d7981 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/7/testReport/ |
| Max. process+thread count | 3217 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1292931733
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 2s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 13s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 6s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 40m 47s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 147m 6s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRBFConfigFields |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a12238d0ccd3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 85bba525dda210214c65bbb8f46d5effc9c9f610 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/9/testReport/ |
| Max. process+thread count | 3437 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on PR #5071:
URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1292926230
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 4s | | trunk passed |
| +1 :green_heart: | compile | 0m 49s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 50s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 26s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 46s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 35m 27s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt) | hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. |
| | | 141m 16s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRBFConfigFields |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/5071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c1fd5e2000d4 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 84c4504e51f5e4885700efd5b53c31a05550ecc2 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/8/testReport/ |
| Max. process+thread count | 3189 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5071/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org