You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/08/29 01:27:24 UTC
[GitHub] [hadoop] abhishekdas99 opened a new pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
abhishekdas99 opened a new pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260
https://issues.apache.org/jira/browse/HADOOP-17028
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-854336249
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 27s | | trunk passed |
| +1 :green_heart: | compile | 28m 53s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 6s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 13s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 55s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | | the patch passed |
| +1 :green_heart: | compile | 22m 7s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 22m 7s | | the patch passed |
| +1 :green_heart: | compile | 19m 16s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 55s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/results-checkstyle-root.txt) | root: The patch generated 7 new + 154 unchanged - 5 fixed = 161 total (was 159) |
| +1 :green_heart: | mvnsite | 3m 2s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 9s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 2m 35s | [/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html) | hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 19m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 55s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 328m 19s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not generate ASF License warnings. |
| | | 553m 43s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of org.apache.hadoop.fs.viewfs.InodeTree$INodeLink.targetFileSystem; locked 42% of time Unsynchronized access at InodeTree.java:42% of time Unsynchronized access at InodeTree.java:[line 312] |
| Failed junit tests | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8e62c2baf159 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c66c2bf58bd32efeb723b23fda3e52566128de18 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/testReport/ |
| Max. process+thread count | 2812 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-687031356
> I like the lazy eval style
>
> * thread safety: what is the story here?
> * what's the story w.r.t IOExceptions being raised in the init function?
>
> in org.apache.hadoop.fs.impl.FunctionsRaisingIOE we have an interface for functions which explicilty raise them. Using that or something like it makes sense here
Thanks @steveloughran for the review.
I have incorporated FunctionsRaisingIOE to take care of IOException. Thanks for the pointer. Also added synchronized block around initialization of file system instance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r483151165
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +290,15 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ public T getTargetFileSystem() throws IOException {
Review comment:
This is now doing more than get. Needs javadocs. What is the thread safety story here?
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +290,15 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ public T getTargetFileSystem() throws IOException {
+ if (isSingleLink && targetFileSystem == null) {
+ targetFileSystem = initFileSystemFunction.apply(targetDirLinkList[0]);
+
+ if (targetFileSystem == null) {
+ throw new IOException("Could not initialize "
+ + "target File System for URI : " + targetDirLinkList[0]);
Review comment:
make it a PathIOException with the given path.
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java
##########
@@ -35,6 +35,7 @@
import java.util.Set;
import com.google.common.base.Preconditions;
+import java.util.function.Function;
Review comment:
import placement
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -30,6 +30,7 @@
import java.util.Map;
import java.util.Map.Entry;
+import java.util.function.Function;
Review comment:
move to java import group
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsConfig.java
##########
@@ -21,6 +21,7 @@
import java.net.URI;
import java.net.URISyntaxException;
+import java.util.function.Function;
Review comment:
import placement
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-683532916
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 18s | trunk passed |
| +1 :green_heart: | compile | 21m 29s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 17m 42s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 49s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | trunk passed |
| +1 :green_heart: | shadedclient | 18m 11s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 24s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 14s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 12s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 20m 11s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 11s | the patch passed |
| +1 :green_heart: | compile | 17m 41s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 41s | the patch passed |
| +1 :green_heart: | checkstyle | 0m 45s | hadoop-common-project/hadoop-common: The patch generated 0 new + 159 unchanged - 4 fixed = 159 total (was 163) |
| +1 :green_heart: | mvnsite | 1m 20s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 41s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 21s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 9m 28s | hadoop-common in the patch passed. |
| -1 :x: | asflicense | 0m 45s | The patch generated 3 ASF License warnings. |
| | | 169m 47s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 02736962aaf8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 56a5c360a10 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 1343 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310790
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -936,8 +956,13 @@ public void setWriteChecksum(final boolean writeChecksum) {
fsState.getMountPoints();
Set<FileSystem> children = new HashSet<FileSystem>();
for (InodeTree.MountPoint<FileSystem> mountPoint : mountPoints) {
- FileSystem targetFs = mountPoint.target.targetFileSystem;
- children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ try {
+ FileSystem targetFs = mountPoint.target.getTargetFileSystem();
+ children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ } catch (IOException ex) {
+ LOG.error("Could not add child filesystems "
+ + "for source path " + mountPoint.src);
Review comment:
Changed the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310547
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
+
+ if (targetDirLinkList.length == 1) {
+ synchronized (this) {
+ targetFileSystem = fileSystemInitFunc.apply(targetDirLinkList[0]);
Review comment:
Added extra check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310313
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
Review comment:
added . at the end of the line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-853374990
@umamaheswararao @steveloughran bumping up the conversation on this thread. Recently we have faced lot of issues (OOM, fs initialization time going up) because of not having lazy initialization. Want to get a review on this one.
cc @virajith @shvachko
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-855186914
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 17s | | trunk passed |
| +1 :green_heart: | compile | 22m 34s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 13s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 15s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 7s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 16s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 44s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 21m 56s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 21m 56s | | the patch passed |
| +1 :green_heart: | compile | 19m 25s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 21s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/artifact/out/results-checkstyle-root.txt) | root: The patch generated 7 new + 154 unchanged - 5 fixed = 161 total (was 159) |
| +1 :green_heart: | mvnsite | 3m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 27s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 2m 53s | [/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html) | hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 17m 30s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 28s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 242m 23s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 6s | | The patch does not generate ASF License warnings. |
| | | 452m 39s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of org.apache.hadoop.fs.viewfs.InodeTree$INodeLink.targetFileSystem; locked 42% of time Unsynchronized access at InodeTree.java:42% of time Unsynchronized access at InodeTree.java:[line 312] |
| Failed junit tests | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 6e2b4aa3c3fe 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 142d2ff9d10d794328727003ca400d6683661e79 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/testReport/ |
| Max. process+thread count | 3084 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao edited a comment on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
umamaheswararao edited a comment on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-687277527
Thanks @steveloughran for review!
In addition to his comments, I have the following question/comment.
In practice, I am thinking this lazy initialization will not give us real benefit in the following case with MR/YARN.
In DelegationTokenIssuer#collectDelegationTokens, it will try to get tokens from all children.
```
// Now collect the tokens from the children.
final DelegationTokenIssuer[] ancillary =
issuer.getAdditionalTokenIssuers();
if (ancillary != null) {
for (DelegationTokenIssuer subIssuer : ancillary) {
collectDelegationTokens(subIssuer, renewer, credentials, tokens);
}
}
```
I am wondering this call will make all fs to be initialized. That means we are not getting this lazy initialization benefit fully.
Did you consider this case?
Is there a way to avoid this ? MR/YARN team can suggest something if we have alternative way to load tokens.
CC: @wangdatan @sunilgovind @rohithsharmaks
CC: @xiaoyuyao
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-683235472
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 20s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 14s | trunk passed |
| +1 :green_heart: | compile | 26m 58s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 23m 15s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 1m 0s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 50s | trunk passed |
| +1 :green_heart: | shadedclient | 20m 15s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 14s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 20m 18s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 18s | the patch passed |
| +1 :green_heart: | compile | 17m 30s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 30s | the patch passed |
| -0 :warning: | checkstyle | 0m 47s | hadoop-common-project/hadoop-common: The patch generated 9 new + 159 unchanged - 4 fixed = 168 total (was 163) |
| +1 :green_heart: | mvnsite | 1m 23s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 16m 0s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 43s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | findbugs | 2m 46s | hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
||| _ Other Tests _ |
| -1 :x: | unit | 10m 18s | hadoop-common in the patch passed. |
| -1 :x: | asflicense | 0m 52s | The patch generated 3 ASF License warnings. |
| | | 189m 1s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-common-project/hadoop-common |
| | uri must be non-null but is marked as nullable At ViewFileSystem.java:is marked as nullable At ViewFileSystem.java:[lines 308-318] |
| | uri must be non-null but is marked as nullable At ViewFs.java:is marked as nullable At ViewFs.java:[lines 246-257] |
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f43b4bba4f97 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 44542863f41 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| findbugs | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 3273 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-683235472
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 20s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 14s | trunk passed |
| +1 :green_heart: | compile | 26m 58s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 23m 15s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 1m 0s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 50s | trunk passed |
| +1 :green_heart: | shadedclient | 20m 15s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 14s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 20m 18s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 20m 18s | the patch passed |
| +1 :green_heart: | compile | 17m 30s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 30s | the patch passed |
| -0 :warning: | checkstyle | 0m 47s | hadoop-common-project/hadoop-common: The patch generated 9 new + 159 unchanged - 4 fixed = 168 total (was 163) |
| +1 :green_heart: | mvnsite | 1m 23s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 16m 0s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 43s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | findbugs | 2m 46s | hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
||| _ Other Tests _ |
| -1 :x: | unit | 10m 18s | hadoop-common in the patch passed. |
| -1 :x: | asflicense | 0m 52s | The patch generated 3 ASF License warnings. |
| | | 189m 1s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-common-project/hadoop-common |
| | uri must be non-null but is marked as nullable At ViewFileSystem.java:is marked as nullable At ViewFileSystem.java:[lines 308-318] |
| | uri must be non-null but is marked as nullable At ViewFs.java:is marked as nullable At ViewFs.java:[lines 246-257] |
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f43b4bba4f97 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 44542863f41 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| findbugs | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 3273 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-856383553
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 35s | | trunk passed |
| +1 :green_heart: | compile | 22m 34s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 4s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 1s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 48s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 11s | | the patch passed |
| +1 :green_heart: | compile | 21m 48s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 21m 48s | | the patch passed |
| +1 :green_heart: | compile | 19m 10s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 58s | | root: The patch generated 0 new + 154 unchanged - 5 fixed = 154 total (was 159) |
| +1 :green_heart: | mvnsite | 2m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 2s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 7s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 17s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 51s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 322m 10s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not generate ASF License warnings. |
| | | 534m 13s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux d4f6e49ceb75 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2f8e9a6b0df557e63ec30f65c171bb0f765f5bc4 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/7/testReport/ |
| Max. process+thread count | 3135 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644407043
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1428,4 +1430,45 @@ public void testGetContentSummaryWithFileInLocalFS() throws Exception {
summaryAfter.getLength());
}
}
+
+ @Test
+ public void testTargetFileSystemLazyInitialization() throws Exception {
+ final String clusterName = "cluster" + new Random().nextInt();
+ Configuration config = new Configuration(conf);
+ config.setBoolean(CONFIG_VIEWFS_ENABLE_INNER_CACHE, false);
+ config.setClass("fs.mockfs.impl",
+ TestChRootedFileSystem.MockFileSystem.class, FileSystem.class);
+ ConfigUtil.addLink(config, clusterName, "/user",
+ URI.create("mockfs://mockauth1/mockpath"));
+ ConfigUtil.addLink(config, clusterName,
+ "/mock", URI.create("mockfs://mockauth/mockpath"));
+
+ final int cacheSize = TestFileUtil.getCacheSize();
Review comment:
You can call `closeAll()` here, which clears the cache completely. And the you can compare with 0 in the assert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 edited a comment on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 edited a comment on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-853374990
@umamaheswararao @steveloughran bumping up the conversation on this thread. Recently we have faced lot of issues (OOM, fs initialization time going up) because of not having lazy initialization.
We see these issues where we have 10k mount points and ViewFileSystem initializes all the mount points during initialization.
Want to get a review on this one.
cc @virajith @shvachko
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r483495855
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +290,15 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ public T getTargetFileSystem() throws IOException {
Review comment:
Added javadoc. Also added synchronized block around apply call to make sure only one thread is initializing the file system instance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-683496861
@umamaheswararao Can you please review this PR when you get a chance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644426716
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -37,6 +37,7 @@
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.UnsupportedFileSystemException;
+import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE;
Review comment:
replaced `org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE` with `org.apache.hadoop.util.functional.FunctionRaisingIOE`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-855189780
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 4s | | trunk passed |
| +1 :green_heart: | compile | 22m 47s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 20s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 11s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 19s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 36s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 51s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 20s | | the patch passed |
| +1 :green_heart: | compile | 22m 18s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 22m 18s | | the patch passed |
| +1 :green_heart: | compile | 19m 18s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 39s | | root: The patch generated 0 new + 154 unchanged - 5 fixed = 154 total (was 159) |
| +1 :green_heart: | mvnsite | 3m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 11s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 26s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 2m 43s | [/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html) | hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 19m 36s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 29s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 238m 50s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not generate ASF License warnings. |
| | | 453m 15s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of org.apache.hadoop.fs.viewfs.InodeTree$INodeLink.targetFileSystem; locked 80% of time Unsynchronized access at InodeTree.java:80% of time Unsynchronized access at InodeTree.java:[line 327] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 72cec98d8f3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7708fdd132ec31f68722eca9c3fb01dfcf21aeaf |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/testReport/ |
| Max. process+thread count | 3073 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 edited a comment on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 edited a comment on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-853374990
@umamaheswararao @steveloughran bumping up the conversation on this thread. Recently we have faced lot of issues (OOM, fs initialization time going up) because of not having lazy initialization.
We see these issues where we have 10k mount points and ViewFileSystem initializes all the mount points during initialization.
Want to get a review on this one.
cc @virajith @shvachko
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko edited a comment on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko edited a comment on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-872643068
Looks good generally. I checked the tests are catching previous errors.
- There seems to be one last checkstyle warning.
- Also I see you use `import static org.junit.Assert.*;` and `viewfs.Constants.*;` We should import only what is actually needed. Avoid *-imports.
- And I found an empty line added in `ViewFileSystem`. Don't know why checkstyle didn't catch it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644393901
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -37,6 +37,7 @@
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.UnsupportedFileSystemException;
+import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE;
Review comment:
It looks like @steveloughran deprecated this class later after his comment. Don't know what prompted the refactoring, but
`import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE`
should be now
`org.apache.hadoop.util.functional.FunctionRaisingIOE`
##########
File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/viewfs/ViewFileSystemBaseTest.java
##########
@@ -1428,4 +1430,45 @@ public void testGetContentSummaryWithFileInLocalFS() throws Exception {
summaryAfter.getLength());
}
}
+
+ @Test
+ public void testTargetFileSystemLazyInitialization() throws Exception {
+ final String clusterName = "cluster" + new Random().nextInt();
+ Configuration config = new Configuration(conf);
+ config.setBoolean(CONFIG_VIEWFS_ENABLE_INNER_CACHE, false);
+ config.setClass("fs.mockfs.impl",
+ TestChRootedFileSystem.MockFileSystem.class, FileSystem.class);
+ ConfigUtil.addLink(config, clusterName, "/user",
+ URI.create("mockfs://mockauth1/mockpath"));
+ ConfigUtil.addLink(config, clusterName,
+ "/mock", URI.create("mockfs://mockauth/mockpath"));
+
+ final int cacheSize = TestFileUtil.getCacheSize();
Review comment:
You can call `closeAll()` here, which clears the cache completely. And the you can compare with 0 in the assert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-871841580
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 7 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 47s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 10s | | trunk passed |
| +1 :green_heart: | compile | 20m 51s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 50s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 20s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 30s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 9s | | the patch passed |
| +1 :green_heart: | compile | 20m 33s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 33s | | the patch passed |
| +1 :green_heart: | compile | 18m 11s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 45s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/9/artifact/out/results-checkstyle-root.txt) | root: The patch generated 1 new + 161 unchanged - 6 fixed = 162 total (was 167) |
| +1 :green_heart: | mvnsite | 3m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 20s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 50s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 57s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 234m 15s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/9/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not generate ASF License warnings. |
| | | 436m 45s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.balancer.TestBalancer |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/9/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7a29c1ed04ad 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a658312ca6e01f0a31d764858360aa52013efb1b |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/9/testReport/ |
| Max. process+thread count | 3095 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310654
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -855,7 +861,11 @@ public void setVerifyChecksum(final boolean verifyChecksum) {
List<InodeTree.MountPoint<FileSystem>> mountPoints =
fsState.getMountPoints();
for (InodeTree.MountPoint<FileSystem> mount : mountPoints) {
- mount.target.targetFileSystem.setVerifyChecksum(verifyChecksum);
+ try {
+ mount.target.getTargetFileSystem().setVerifyChecksum(verifyChecksum);
+ } catch (IOException ex) {
+ LOG.error("Could not set verifyChecksum for source path " + mount.src);
+ }
Review comment:
Changed the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-687030524
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | patch | 0m 4s | https://github.com/apache/hadoop/pull/2260 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/4/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-855279818
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 52s | | trunk passed |
| +1 :green_heart: | compile | 22m 36s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 15s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 7s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 2s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 11s | | the patch passed |
| +1 :green_heart: | compile | 21m 58s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 21m 58s | | the patch passed |
| +1 :green_heart: | compile | 19m 15s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 57s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/6/artifact/out/results-checkstyle-root.txt) | root: The patch generated 1 new + 154 unchanged - 5 fixed = 155 total (was 159) |
| +1 :green_heart: | mvnsite | 3m 2s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 2s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 6s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 17s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 53s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 333m 19s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not generate ASF License warnings. |
| | | 546m 29s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/6/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 80765823aeb8 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d8aac169b6c1203efaefdb2069dc9c4d1286c454 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/6/testReport/ |
| Max. process+thread count | 3065 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644393901
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -37,6 +37,7 @@
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.UnsupportedFileSystemException;
+import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE;
Review comment:
It looks like @steveloughran deprecated this class later after his comment. Don't know what prompted the refactoring, but
`import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE`
should be now
`org.apache.hadoop.util.functional.FunctionRaisingIOE`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310415
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-683389116
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 28m 51s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 7s | trunk passed |
| +1 :green_heart: | compile | 19m 37s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 16m 53s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 55s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 38s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 39s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 34s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 17s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 14s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | the patch passed |
| +1 :green_heart: | compile | 18m 47s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 18m 47s | the patch passed |
| +1 :green_heart: | compile | 17m 38s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | javac | 17m 38s | the patch passed |
| -0 :warning: | checkstyle | 0m 51s | hadoop-common-project/hadoop-common: The patch generated 5 new + 159 unchanged - 4 fixed = 164 total (was 163) |
| +1 :green_heart: | mvnsite | 1m 26s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 4s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 38s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | findbugs | 2m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 26s | hadoop-common in the patch passed. |
| -1 :x: | asflicense | 0m 56s | The patch generated 3 ASF License warnings. |
| | | 189m 1s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux e7ebdc6666bf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 56a5c360a10 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/2/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/2/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/2/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 1592 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-853374990
@umamaheswararao @steveloughran bumping up the conversation on this thread. Recently we have faced lot of issues (OOM, fs initialization time going up) because of not having lazy initialization. Want to get a review on this one.
cc @virajith @shvachko
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-878757325
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 7 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 12s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 28s | | trunk passed |
| +1 :green_heart: | compile | 20m 59s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 12s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 17s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 19s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 49s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 7s | | the patch passed |
| +1 :green_heart: | compile | 20m 14s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 14s | | the patch passed |
| +1 :green_heart: | compile | 18m 20s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 43s | | root: The patch generated 0 new + 161 unchanged - 6 fixed = 161 total (was 167) |
| +1 :green_heart: | mvnsite | 3m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 29s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 13s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 37s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 241m 23s | | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not generate ASF License warnings. |
| | | 446m 51s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/10/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7c12e38627e0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 541e432b56f07c775db81f472cb28f0f1b4bdca6 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/10/testReport/ |
| Max. process+thread count | 3216 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-878824466
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 7 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 24s | | trunk passed |
| +1 :green_heart: | compile | 24m 30s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 20m 20s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 13s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 31s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 26s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 24s | | the patch passed |
| +1 :green_heart: | compile | 23m 18s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 23m 18s | | the patch passed |
| +1 :green_heart: | compile | 19m 44s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 4m 1s | | root: The patch generated 0 new + 161 unchanged - 6 fixed = 161 total (was 167) |
| +1 :green_heart: | mvnsite | 3m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 21s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 35s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 28s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 19m 13s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 234m 31s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 8s | | The patch does not generate ASF License warnings. |
| | | 458m 7s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/11/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e510f81bf31b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 824c4c025c21e7ea1072dec9558529c3806f52f7 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/11/testReport/ |
| Max. process+thread count | 2740 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r483645622
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
Review comment:
nit: add a . to keep javadoc happy
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
Review comment:
nit, curly { }
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
+
+ if (targetDirLinkList.length == 1) {
+ synchronized (this) {
+ targetFileSystem = fileSystemInitFunc.apply(targetDirLinkList[0]);
Review comment:
this will dual init as thread #2 will block. You need another check inside the sync block so the second thread won't repeat itself
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -893,6 +906,9 @@ public short getDefaultReplication(Path f) {
return res.targetFileSystem.getDefaultReplication(res.remainingPath);
} catch (FileNotFoundException e) {
throw new NotInMountpointException(f, "getDefaultReplication");
+ } catch (IOException e) {
+ throw new RuntimeException("Not able to initialize fs in "
Review comment:
FYI. I've got a WrappedIOException, but in #2069 making it a public `org.apache.hadoop.fs.functional.RuntimeIOException` whose cause is only ever IOE. Not something to be picked up here (yet), but worth knowing. I'm trying to make the functional & lambda/expression stuff more useable,
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -855,7 +861,11 @@ public void setVerifyChecksum(final boolean verifyChecksum) {
List<InodeTree.MountPoint<FileSystem>> mountPoints =
fsState.getMountPoints();
for (InodeTree.MountPoint<FileSystem> mount : mountPoints) {
- mount.target.targetFileSystem.setVerifyChecksum(verifyChecksum);
+ try {
+ mount.target.getTargetFileSystem().setVerifyChecksum(verifyChecksum);
+ } catch (IOException ex) {
+ LOG.error("Could not set verifyChecksum for source path " + mount.src);
+ }
Review comment:
log the full stack trace
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -936,8 +956,13 @@ public void setWriteChecksum(final boolean writeChecksum) {
fsState.getMountPoints();
Set<FileSystem> children = new HashSet<FileSystem>();
for (InodeTree.MountPoint<FileSystem> mountPoint : mountPoints) {
- FileSystem targetFs = mountPoint.target.targetFileSystem;
- children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ try {
+ FileSystem targetFs = mountPoint.target.getTargetFileSystem();
+ children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ } catch (IOException ex) {
+ LOG.error("Could not add child filesystems "
+ + "for source path " + mountPoint.src);
Review comment:
+full exception log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-687080093
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 40s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 4s | trunk passed |
| +1 :green_heart: | compile | 22m 17s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 18m 21s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +1 :green_heart: | checkstyle | 0m 49s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | trunk passed |
| +1 :green_heart: | shadedclient | 18m 7s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 31s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 21s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| +0 :ok: | spotbugs | 2m 18s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 16s | trunk passed |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 29s | hadoop-common in the patch failed. |
| -1 :x: | compile | 1m 1s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| -1 :x: | javac | 1m 1s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| -1 :x: | compile | 0m 50s | root in the patch failed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| -1 :x: | javac | 0m 50s | root in the patch failed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01. |
| -0 :warning: | checkstyle | 0m 33s | hadoop-common-project/hadoop-common: The patch generated 5 new + 159 unchanged - 4 fixed = 164 total (was 163) |
| -1 :x: | mvnsite | 0m 31s | hadoop-common in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| -1 :x: | shadedclient | 0m 40s | patch has errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 14s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 1m 12s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| -1 :x: | findbugs | 0m 28s | hadoop-common in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 29s | hadoop-common in the patch failed. |
| +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. |
| | | 105m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 49a2b68f61f9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 696e4fe50e4 |
| Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 |
| mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-mvninstall-hadoop-common-project_hadoop-common.txt |
| compile | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-compile-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| javac | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-compile-root-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| compile | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt |
| javac | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01.txt |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt |
| mvnsite | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-mvnsite-hadoop-common-project_hadoop-common.txt |
| findbugs | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-findbugs-hadoop-common-project_hadoop-common.txt |
| unit | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/5/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on a change in pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on a change in pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#discussion_r644310313
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
Review comment:
added . at the end of the line
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
Review comment:
Done
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -284,7 +288,20 @@ boolean isInternalDir() {
return false;
}
- public T getTargetFileSystem() {
+ /**
+ * Gets lazily loaded instance of FileSystem
+ * @return An Initialized instance of T
+ * @throws IOException
+ */
+ public T getTargetFileSystem() throws IOException {
+ if (targetFileSystem != null)
+ return targetFileSystem;
+
+ if (targetDirLinkList.length == 1) {
+ synchronized (this) {
+ targetFileSystem = fileSystemInitFunc.apply(targetDirLinkList[0]);
Review comment:
Added extra check
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -855,7 +861,11 @@ public void setVerifyChecksum(final boolean verifyChecksum) {
List<InodeTree.MountPoint<FileSystem>> mountPoints =
fsState.getMountPoints();
for (InodeTree.MountPoint<FileSystem> mount : mountPoints) {
- mount.target.targetFileSystem.setVerifyChecksum(verifyChecksum);
+ try {
+ mount.target.getTargetFileSystem().setVerifyChecksum(verifyChecksum);
+ } catch (IOException ex) {
+ LOG.error("Could not set verifyChecksum for source path " + mount.src);
+ }
Review comment:
Changed the code.
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##########
@@ -936,8 +956,13 @@ public void setWriteChecksum(final boolean writeChecksum) {
fsState.getMountPoints();
Set<FileSystem> children = new HashSet<FileSystem>();
for (InodeTree.MountPoint<FileSystem> mountPoint : mountPoints) {
- FileSystem targetFs = mountPoint.target.targetFileSystem;
- children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ try {
+ FileSystem targetFs = mountPoint.target.getTargetFileSystem();
+ children.addAll(Arrays.asList(targetFs.getChildFileSystems()));
+ } catch (IOException ex) {
+ LOG.error("Could not add child filesystems "
+ + "for source path " + mountPoint.src);
Review comment:
Changed the code.
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/InodeTree.java
##########
@@ -37,6 +37,7 @@
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.UnsupportedFileSystemException;
+import org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE;
Review comment:
replaced `org.apache.hadoop.fs.impl.FunctionsRaisingIOE.FunctionRaisingIOE` with `org.apache.hadoop.util.functional.FunctionRaisingIOE`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-872643068
Looks good generally. I checked the tests are catching previous errors.
- There seems to be one last checkstyle warning.
- Also I see you use {{import static org.junit.Assert.*;}} and {{viewfs.Constants.*;}} We should import only what is actually needed. Avoid *-imports.
- And I found an empty line added in {{ViewFileSystem}}. Don't know why checkstyle didn't catch it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-854336249
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 5 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 27s | | trunk passed |
| +1 :green_heart: | compile | 28m 53s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 34s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 6s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 13s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 55s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | | the patch passed |
| +1 :green_heart: | compile | 22m 7s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 22m 7s | | the patch passed |
| +1 :green_heart: | compile | 19m 16s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 55s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/results-checkstyle-root.txt) | root: The patch generated 7 new + 154 unchanged - 5 fixed = 161 total (was 159) |
| +1 :green_heart: | mvnsite | 3m 2s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 5s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 9s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 2m 35s | [/new-spotbugs-hadoop-common-project_hadoop-common.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/new-spotbugs-hadoop-common-project_hadoop-common.html) | hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 19m 37s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 55s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 328m 19s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not generate ASF License warnings. |
| | | 553m 43s | | |
| Reason | Tests |
|-------:|:------|
| SpotBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of org.apache.hadoop.fs.viewfs.InodeTree$INodeLink.targetFileSystem; locked 42% of time Unsynchronized access at InodeTree.java:42% of time Unsynchronized access at InodeTree.java:[line 312] |
| Failed junit tests | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8e62c2baf159 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / c66c2bf58bd32efeb723b23fda3e52566128de18 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/testReport/ |
| Max. process+thread count | 2812 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-881306433
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 0s | | Docker mode activated. |
| -1 :x: | patch | 0m 26s | | https://github.com/apache/hadoop/pull/2260 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/12/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] umamaheswararao commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
umamaheswararao commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-687277527
Thanks @steveloughran for review!
In addition to his comments, I have the following question/comment.
In practice, I am thinking this lazy initialization will not give us real benefit in the following case with MR/YARN.
In DelegationTokenIssuer#collectDelegationTokens, it will try to get tokens from all children.
```
// Now collect the tokens from the children.
final DelegationTokenIssuer[] ancillary =
issuer.getAdditionalTokenIssuers();
if (ancillary != null) {
for (DelegationTokenIssuer subIssuer : ancillary) {
collectDelegationTokens(subIssuer, renewer, credentials, tokens);
}
}
```
I am wondering this call make all fs to be initialized. That means we are not getting this lazy initialization benefit fully.
Did you consider this case?
Is there a way to avoid this ? MR/YAN team can suggest something if we have alternative way to load tokens.
CC: @wangdatan @sunilgovind @rohithsharmaks
CC: @xiaoyuyao
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] abhishekdas99 commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
abhishekdas99 commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-878573343
> Looks good generally. I checked the tests are catching previous errors.
>
> * There seems to be one last checkstyle warning.
> * Also I see you use `import static org.junit.Assert.*;` and `viewfs.Constants.*;` We should import only what is actually needed. Avoid *-imports.
> * And I found an empty line added in `ViewFileSystem`. Don't know why checkstyle didn't catch it.
@shvachko I have removed the checkstyle warning. Also added the individual imports and removed the empty line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] shvachko commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
shvachko commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-878773959
+1 on the latest patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2260: HADOOP-17028. ViewFS should initialize mounted target filesystems lazily
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2260:
URL: https://github.com/apache/hadoop/pull/2260#issuecomment-871286026
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 39s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 7 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 52s | | trunk passed |
| +1 :green_heart: | compile | 31m 13s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 24m 55s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 6s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 57s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 26s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 18s | | the patch passed |
| +1 :green_heart: | compile | 23m 5s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 23m 5s | | the patch passed |
| +1 :green_heart: | compile | 19m 21s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 57s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/8/artifact/out/results-checkstyle-root.txt) | root: The patch generated 7 new + 161 unchanged - 6 fixed = 168 total (was 167) |
| +1 :green_heart: | mvnsite | 3m 1s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 6s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 7s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 26s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 48s | | hadoop-common in the patch passed. |
| -1 :x: | unit | 337m 5s | [/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt) | hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not generate ASF License warnings. |
| | | 573m 46s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.TestDFSShell |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor |
| | hadoop.hdfs.server.namenode.ha.TestBootstrapStandby |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2260 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 81c68a4e9538 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8fa62c8b6d53c75616504acd570389e43d547a14 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/8/testReport/ |
| Max. process+thread count | 2462 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2260/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org