You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/06/07 03:00:36 UTC
[GitHub] [hadoop] abhishekdas99 opened a new pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multi…
abhishekdas99 opened a new pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060
…ple children mountpoints pointing to different filesystems
## NOTICE
Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multi…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-640160994
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 22m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 20s | trunk passed |
| +1 :green_heart: | compile | 16m 59s | trunk passed |
| +1 :green_heart: | checkstyle | 0m 56s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 28s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 44s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 3s | trunk passed |
| +0 :ok: | spotbugs | 2m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 7s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 16m 23s | the patch passed |
| +1 :green_heart: | javac | 16m 23s | the patch passed |
| -0 :warning: | checkstyle | 0m 54s | hadoop-common-project/hadoop-common: The patch generated 14 new + 103 unchanged - 1 fixed = 117 total (was 104) |
| +1 :green_heart: | mvnsite | 1m 28s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 37s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 5s | the patch passed |
| +1 :green_heart: | findbugs | 2m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 43s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 129m 30s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a64f097e4fcf 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 3ca15292c55 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/1/testReport/ |
| Max. process+thread count | 1658 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-652077114
Thanks @ayushtkn for the review. I have addressed your comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-652123187
Thanx, Some checkstyle warnings surfaced, Can you check them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multi…
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-640175270
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 22m 23s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 26s | trunk passed |
| +1 :green_heart: | compile | 17m 1s | trunk passed |
| +1 :green_heart: | checkstyle | 0m 55s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 25s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 5s | trunk passed |
| +0 :ok: | spotbugs | 2m 10s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 8s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 16m 14s | the patch passed |
| +1 :green_heart: | javac | 16m 14s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 54s | hadoop-common-project/hadoop-common: The patch generated 0 new + 103 unchanged - 1 fixed = 103 total (was 104) |
| +1 :green_heart: | mvnsite | 1m 25s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 48s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | the patch passed |
| +1 :green_heart: | findbugs | 2m 13s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 23s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 55s | The patch does not generate ASF License warnings. |
| | | 128m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f2aaadb054ba 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 3ca15292c55 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/2/testReport/ |
| Max. process+thread count | 1371 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a change in pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on a change in pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#discussion_r447990862
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1369,4 +1381,61 @@ public void testDeleteOnExit() throws Exception {
viewFs.close();
assertFalse(fsTarget.exists(realTestPath));
}
+
+ @Test
+ public void testGetContentSummary() throws IOException {
+ ContentSummary summaryBefore =
+ fsView.getContentSummary(new Path("/internalDir"));
+ String expected = "GET CONTENT SUMMARY";
+ Path filePath =
+ new Path("/internalDir/internalDir2/linkToDir3", "foo");
+
+ try (FSDataOutputStream outputStream = fsView.create(filePath)) {
+ try (OutputStreamWriter writer =
+ new OutputStreamWriter(outputStream, StandardCharsets.UTF_8)) {
+ try (BufferedWriter buffer = new BufferedWriter(writer)) {
+ buffer.write(expected);
+ }
Review comment:
Will doing just this not work?
```
outputStream.write(expected.getBytes());
```
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1369,4 +1381,61 @@ public void testDeleteOnExit() throws Exception {
viewFs.close();
assertFalse(fsTarget.exists(realTestPath));
}
+
+ @Test
+ public void testGetContentSummary() throws IOException {
+ ContentSummary summaryBefore =
+ fsView.getContentSummary(new Path("/internalDir"));
+ String expected = "GET CONTENT SUMMARY";
+ Path filePath =
+ new Path("/internalDir/internalDir2/linkToDir3", "foo");
+
+ try (FSDataOutputStream outputStream = fsView.create(filePath)) {
+ try (OutputStreamWriter writer =
+ new OutputStreamWriter(outputStream, StandardCharsets.UTF_8)) {
+ try (BufferedWriter buffer = new BufferedWriter(writer)) {
+ buffer.write(expected);
+ }
+ }
+ }
+
+ Path newDirPath = new Path("/internalDir/linkToDir2", "bar");
+ fsView.mkdirs(newDirPath);
+
+ ContentSummary summaryAfter =
+ fsView.getContentSummary(new Path("/internalDir"));
+ Assert.assertEquals("The file count didn't match",
+ summaryBefore.getFileCount() + 1,
+ summaryAfter.getFileCount());
+ Assert.assertEquals("The size didn't match",
+ summaryBefore.getLength() + expected.length(),
+ summaryAfter.getLength());
+ Assert.assertEquals("The directory count didn't match",
Review comment:
nit: There is already a static import -
```
import static org.junit.Assert.*;
```
no need to have `Assert.`
Similarly for the below test aswell
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-652126077
> Thanx, Some checkstyle warnings surfaced, Can you check them.
Fixed the check style issues
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#discussion_r448011766
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1369,4 +1381,61 @@ public void testDeleteOnExit() throws Exception {
viewFs.close();
assertFalse(fsTarget.exists(realTestPath));
}
+
+ @Test
+ public void testGetContentSummary() throws IOException {
+ ContentSummary summaryBefore =
+ fsView.getContentSummary(new Path("/internalDir"));
+ String expected = "GET CONTENT SUMMARY";
+ Path filePath =
+ new Path("/internalDir/internalDir2/linkToDir3", "foo");
+
+ try (FSDataOutputStream outputStream = fsView.create(filePath)) {
+ try (OutputStreamWriter writer =
+ new OutputStreamWriter(outputStream, StandardCharsets.UTF_8)) {
+ try (BufferedWriter buffer = new BufferedWriter(writer)) {
+ buffer.write(expected);
+ }
+ }
+ }
+
+ Path newDirPath = new Path("/internalDir/linkToDir2", "bar");
+ fsView.mkdirs(newDirPath);
+
+ ContentSummary summaryAfter =
+ fsView.getContentSummary(new Path("/internalDir"));
+ Assert.assertEquals("The file count didn't match",
+ summaryBefore.getFileCount() + 1,
+ summaryAfter.getFileCount());
+ Assert.assertEquals("The size didn't match",
+ summaryBefore.getLength() + expected.length(),
+ summaryAfter.getLength());
+ Assert.assertEquals("The directory count didn't match",
Review comment:
Removed `Assert.`
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1369,4 +1381,61 @@ public void testDeleteOnExit() throws Exception {
viewFs.close();
assertFalse(fsTarget.exists(realTestPath));
}
+
+ @Test
+ public void testGetContentSummary() throws IOException {
+ ContentSummary summaryBefore =
+ fsView.getContentSummary(new Path("/internalDir"));
+ String expected = "GET CONTENT SUMMARY";
+ Path filePath =
+ new Path("/internalDir/internalDir2/linkToDir3", "foo");
+
+ try (FSDataOutputStream outputStream = fsView.create(filePath)) {
+ try (OutputStreamWriter writer =
+ new OutputStreamWriter(outputStream, StandardCharsets.UTF_8)) {
+ try (BufferedWriter buffer = new BufferedWriter(writer)) {
+ buffer.write(expected);
+ }
Review comment:
Changed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-651378095
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 18m 47s | trunk passed |
| +1 :green_heart: | compile | 19m 21s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 16m 43s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 54s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | trunk passed |
| +1 :green_heart: | shadedclient | 17m 34s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 38s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 0m 58s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 2m 15s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 13s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 48s | the patch passed |
| +1 :green_heart: | compile | 21m 57s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 21m 57s | the patch passed |
| +1 :green_heart: | compile | 17m 36s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 17m 36s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 54s | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 57s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 45s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 3s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 2m 15s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 30s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. |
| | | 152m 37s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 15ee984481c8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 74fc13cf918 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/4/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/4/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/4/testReport/ |
| Max. process+thread count | 2522 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/4/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#discussion_r447154909
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -850,9 +850,27 @@ public FsServerDefaults getServerDefaults(Path f) throws IOException {
@Override
public ContentSummary getContentSummary(Path f) throws IOException {
+ return getContentSummaryInternal(f);
+ }
+
+ private ContentSummary getContentSummaryInternal(Path f) throws IOException {
InodeTree.ResolveResult<FileSystem> res =
- fsState.resolve(getUriPath(f), true);
- return res.targetFileSystem.getContentSummary(res.remainingPath);
+ fsState.resolve(getUriPath(f), true);
+ if (!res.isInternalDir()) {
Review comment:
@ayushtkn Thanks for the suggestion. I have moved the logic to `InternalDirOfViewFs`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-652122701
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 1s | trunk passed |
| +1 :green_heart: | compile | 19m 42s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 17m 21s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 50s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 23s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 13s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 44s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 2s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 2m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 6s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | the patch passed |
| +1 :green_heart: | compile | 18m 50s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 18m 50s | the patch passed |
| +1 :green_heart: | compile | 17m 12s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 17m 12s | the patch passed |
| -0 :warning: | checkstyle | 0m 57s | hadoop-common-project/hadoop-common: The patch generated 3 new + 103 unchanged - 0 fixed = 106 total (was 103) |
| +1 :green_heart: | mvnsite | 1m 31s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 29s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 43s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 2s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 2m 14s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 22s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 149m 27s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 23f2e5bf5406 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / e8dc862d385 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/testReport/ |
| Max. process+thread count | 1683 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/5/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-652184149
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 21m 33s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 1s | trunk passed |
| +1 :green_heart: | compile | 19m 43s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 17m 16s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 48s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | trunk passed |
| +1 :green_heart: | shadedclient | 20m 37s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 43s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 7s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 2m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 32s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 59s | the patch passed |
| +1 :green_heart: | compile | 25m 9s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 25m 9s | the patch passed |
| +1 :green_heart: | compile | 22m 41s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 22m 41s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 58s | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 18m 12s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 47s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 4s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 2m 58s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 36s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. |
| | | 193m 9s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 23e32fdacb1c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / e8dc862d385 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/6/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/6/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/6/testReport/ |
| Max. process+thread count | 2738 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/6/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a change in pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
ayushtkn commented on a change in pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#discussion_r443502486
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -850,9 +850,27 @@ public FsServerDefaults getServerDefaults(Path f) throws IOException {
@Override
public ContentSummary getContentSummary(Path f) throws IOException {
+ return getContentSummaryInternal(f);
+ }
+
+ private ContentSummary getContentSummaryInternal(Path f) throws IOException {
InodeTree.ResolveResult<FileSystem> res =
- fsState.resolve(getUriPath(f), true);
- return res.targetFileSystem.getContentSummary(res.remainingPath);
+ fsState.resolve(getUriPath(f), true);
+ if (!res.isInternalDir()) {
Review comment:
The logics here are introduced for InternalDirOfViewFs,
I feel rather than touching the normal API, you can implement getContentSummary() in `InternalDirOfViewFs` and have your logic there, That is how all API's which have specific logic for Internal Links are handling. Lets keep the design that way only.
The only thing that you would need there is fsState which isn't there as of now, that you can pass through the constructor, In HDFS-15427, it is being passed, so you can utilize that fsState and independently implement it in `InternalDirOfViewFs`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] ayushtkn merged pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
ayushtkn merged pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2060: HADOOP-17032. Fix getContentSummary in ViewFileSystem to handle multiple children mountpoints pointing to different filesystems
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2060:
URL: https://github.com/apache/hadoop/pull/2060#issuecomment-651115869
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 31s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 7s | trunk passed |
| +1 :green_heart: | compile | 19m 27s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 16m 48s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 53s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 28s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 28s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 44s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 4s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 2m 8s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 7s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 18m 44s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 18m 44s | the patch passed |
| +1 :green_heart: | compile | 17m 5s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 17m 5s | the patch passed |
| -0 :warning: | checkstyle | 0m 52s | hadoop-common-project/hadoop-common: The patch generated 3 new + 103 unchanged - 0 fixed = 106 total (was 103) |
| +1 :green_heart: | mvnsite | 1m 28s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 48s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 34s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 1m 2s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 2m 18s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 34s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 53s | The patch does not generate ASF License warnings. |
| | | 147m 52s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2060 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c37f16d572cf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / c71ce7ac337 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/testReport/ |
| Max. process+thread count | 1785 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2060/3/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org