You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/06/03 12:22:55 UTC
[GitHub] [hadoop] zhaomin1423 opened a new pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
zhaomin1423 opened a new pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071
more information:
https://issues.apache.org/jira/browse/HADOOP-17742#
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-853988493
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 6s | | trunk passed |
| +1 :green_heart: | compile | 22m 16s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 42s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 30s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 21m 52s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 21m 52s | | the patch passed |
| +1 :green_heart: | compile | 19m 22s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 52s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/results-checkstyle-root.txt) | root: The patch generated 4 new + 84 unchanged - 3 fixed = 88 total (was 87) |
| +1 :green_heart: | mvnsite | 2m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 20s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 54s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 3s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 22m 35s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not generate ASF License warnings. |
| | | 224m 7s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 4368ce4cc012 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4f6f65cc09fadbca84e2f56b99555255c7bf13ea |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/testReport/ |
| Max. process+thread count | 2118 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#discussion_r645020297
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
##########
@@ -191,4 +191,8 @@ private DistCpConstants() {
public static final String CLASS_INSTANTIATION_ERROR_MSG =
"Unable to instantiate ";
+
+ public static final String TARGET_TEMP_FILE_PREFIX_COMMA = ".distcp.tmp.";
+
+ public static final String TARGET_TEMP_FILE_PREFIX = "distcp.tmp.";
Review comment:
add _FTP and javadocs to explain why it is needed
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
##########
@@ -191,4 +191,8 @@ private DistCpConstants() {
public static final String CLASS_INSTANTIATION_ERROR_MSG =
"Unable to instantiate ";
+
+ public static final String TARGET_TEMP_FILE_PREFIX_COMMA = ".distcp.tmp.";
Review comment:
1. use DOT instead of COMMA here.
2. add javadoc to explain what it does
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
##########
@@ -662,4 +662,20 @@ public static Path getSplitChunkPath(Path targetFile,
+ ".____distcpSplit____" + srcFileStatus.getChunkOffset()
+ "." + srcFileStatus.getChunkLength());
}
+
+ /**
+ * Return the target temp file prefix
Review comment:
add . on javadoc sentence to keep javadoc happy
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##########
@@ -257,17 +257,18 @@ private void promoteTmpToTarget(Path tmpTarget, Path target, FileSystem fs)
}
}
- private Path getTempFile(Path target, Mapper.Context context) {
+ private Path getTempFile(Path target, Mapper.Context context, FileSystem fileSystem) {
Path targetWorkPath = new Path(context.getConfiguration().
get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));
Path root = target.equals(targetWorkPath) ? targetWorkPath.getParent()
: targetWorkPath;
- Path tempFile = new Path(root, ".distcp.tmp." +
+ String tempFilePrefix = DistCpUtils.getTargetTempFilePrefix(fileSystem);
Review comment:
assuming the Path is resolved, the FS Scheme is that of Path.toURI.getScheme(); no need to pass the FS around.
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
##########
@@ -170,8 +170,9 @@ private void deleteAttemptTempFiles(Path targetWorkPath,
return;
}
+ String tempFilePrefix = DistCpUtils.getTargetTempFilePrefix(targetFS);
FileStatus[] tempFiles = targetFS.globStatus(
- new Path(targetWorkPath, ".distcp.tmp." + jobId.replaceAll("job","attempt") + "*"));
+ new Path(targetWorkPath, tempFilePrefix + jobId.replaceAll("job","attempt") + "*"));
Review comment:
probably need to cut the line down to keep it under 80 chars width
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##########
@@ -129,16 +129,15 @@ private long doCopy(CopyListingFileStatus source, Path target,
throws IOException {
LOG.info("Copying {} to {}", source.getPath(), target);
+ final Configuration configuration = context.getConfiguration();
Review comment:
If you just get the schema in getTempFile() from the path, no need to pass in FileSystem, so no need to move this up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-917327382
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 16s | | trunk passed |
| +1 :green_heart: | compile | 21m 13s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 30s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 51s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 23s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 41s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 15m 5s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 37s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 20m 29s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 29s | | the patch passed |
| +1 :green_heart: | compile | 18m 33s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 33s | | root: The patch generated 0 new + 36 unchanged - 1 fixed = 36 total (was 37) |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 51s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 22s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 42s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 7s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 19m 53s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not generate ASF License warnings. |
| | | 215m 9s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 1e8b453381e6 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dbce5a20beb5c9160c44570b59ed6cfbf40772ec |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/testReport/ |
| Max. process+thread count | 3152 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhaomin1423 commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-854033960
@steveloughran hi, would you help me review the commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-937542901
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | | trunk passed |
| +1 :green_heart: | compile | 21m 26s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 29s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 55s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 23s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 31s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 19m 55s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 20m 45s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 45s | | the patch passed |
| +1 :green_heart: | compile | 18m 45s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 31s | | root: The patch generated 0 new + 35 unchanged - 1 fixed = 35 total (was 36) |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 26s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 29s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 44m 47s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. |
| | | 250m 21s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8f6007b7f7af 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dbce5a20beb5c9160c44570b59ed6cfbf40772ec |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/testReport/ |
| Max. process+thread count | 2181 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#discussion_r645020297
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
##########
@@ -191,4 +191,8 @@ private DistCpConstants() {
public static final String CLASS_INSTANTIATION_ERROR_MSG =
"Unable to instantiate ";
+
+ public static final String TARGET_TEMP_FILE_PREFIX_COMMA = ".distcp.tmp.";
+
+ public static final String TARGET_TEMP_FILE_PREFIX = "distcp.tmp.";
Review comment:
add _FTP and javadocs to explain why it is needed
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
##########
@@ -191,4 +191,8 @@ private DistCpConstants() {
public static final String CLASS_INSTANTIATION_ERROR_MSG =
"Unable to instantiate ";
+
+ public static final String TARGET_TEMP_FILE_PREFIX_COMMA = ".distcp.tmp.";
Review comment:
1. use DOT instead of COMMA here.
2. add javadoc to explain what it does
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
##########
@@ -662,4 +662,20 @@ public static Path getSplitChunkPath(Path targetFile,
+ ".____distcpSplit____" + srcFileStatus.getChunkOffset()
+ "." + srcFileStatus.getChunkLength());
}
+
+ /**
+ * Return the target temp file prefix
Review comment:
add . on javadoc sentence to keep javadoc happy
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##########
@@ -257,17 +257,18 @@ private void promoteTmpToTarget(Path tmpTarget, Path target, FileSystem fs)
}
}
- private Path getTempFile(Path target, Mapper.Context context) {
+ private Path getTempFile(Path target, Mapper.Context context, FileSystem fileSystem) {
Path targetWorkPath = new Path(context.getConfiguration().
get(DistCpConstants.CONF_LABEL_TARGET_WORK_PATH));
Path root = target.equals(targetWorkPath) ? targetWorkPath.getParent()
: targetWorkPath;
- Path tempFile = new Path(root, ".distcp.tmp." +
+ String tempFilePrefix = DistCpUtils.getTargetTempFilePrefix(fileSystem);
Review comment:
assuming the Path is resolved, the FS Scheme is that of Path.toURI.getScheme(); no need to pass the FS around.
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
##########
@@ -170,8 +170,9 @@ private void deleteAttemptTempFiles(Path targetWorkPath,
return;
}
+ String tempFilePrefix = DistCpUtils.getTargetTempFilePrefix(targetFS);
FileStatus[] tempFiles = targetFS.globStatus(
- new Path(targetWorkPath, ".distcp.tmp." + jobId.replaceAll("job","attempt") + "*"));
+ new Path(targetWorkPath, tempFilePrefix + jobId.replaceAll("job","attempt") + "*"));
Review comment:
probably need to cut the line down to keep it under 80 chars width
##########
File path: hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##########
@@ -129,16 +129,15 @@ private long doCopy(CopyListingFileStatus source, Path target,
throws IOException {
LOG.info("Copying {} to {}", source.getPath(), target);
+ final Configuration configuration = context.getConfiguration();
Review comment:
If you just get the schema in getTempFile() from the path, no need to pass in FileSystem, so no need to move this up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-853988493
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 6s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 6s | | trunk passed |
| +1 :green_heart: | compile | 22m 16s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 42s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 30s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 21m 52s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 21m 52s | | the patch passed |
| +1 :green_heart: | compile | 19m 22s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 3m 52s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/results-checkstyle-root.txt) | root: The patch generated 4 new + 84 unchanged - 3 fixed = 88 total (was 87) |
| +1 :green_heart: | mvnsite | 2m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 20s | | the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 54s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 3s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 22m 35s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not generate ASF License warnings. |
| | | 224m 7s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 4368ce4cc012 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4f6f65cc09fadbca84e2f56b99555255c7bf13ea |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/testReport/ |
| Max. process+thread count | 2118 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhaomin1423 commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-854630794
> I've commented on the distcp side of the changes,.
>
> Regarding FTP
>
> 1. That Test suite MUST be restored. That's where the entire semantics of rename in the FS are validated.
> 2. Looking at the history of the code you are cutting, it went in because FTP couldn't rename files into different directories ([HADOOP-9361](https://issues.apache.org/jira/browse/HADOOP-9361)).
>
> What has changed since then?
>
> AFAIK,. the FTP API used by the client has to change to the directory and rename by short name only, not the full the path.
Thanks, I will update the distCp side of the changes.
Regarding FTP, the test suite could be need to modify. After my test using commons-net:3.6, the rename can be work using the full path, just only have the parent directory, and don't have the limit of source path and target path have common parent directory. So, I think the rename method of FTPFilesystem should be modified so that it has the same behavior as other file systems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-856454682
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 28s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 48s | | trunk passed |
| +1 :green_heart: | compile | 22m 46s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 8s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 5s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 40s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 17m 1s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 21m 50s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 21m 50s | | the patch passed |
| +1 :green_heart: | compile | 19m 6s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 19m 6s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| -0 :warning: | checkstyle | 4m 0s | [/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/2/artifact/out/results-checkstyle-root.txt) | root: The patch generated 2 new + 83 unchanged - 3 fixed = 85 total (was 86) |
| +1 :green_heart: | mvnsite | 2m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 7s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 40s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 16m 52s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 40m 3s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not generate ASF License warnings. |
| | | 241m 40s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux f1246607cf7d 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dbce5a20beb5c9160c44570b59ed6cfbf40772ec |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/2/testReport/ |
| Max. process+thread count | 1817 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] zhaomin1423 commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
zhaomin1423 commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-854033960
@steveloughran hi, would you help me review the commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #3071: HADOOP-17742. fix distcp fail when copying to ftp filesystem
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #3071:
URL: https://github.com/apache/hadoop/pull/3071#issuecomment-937542901
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | | trunk passed |
| +1 :green_heart: | compile | 21m 26s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 29s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 55s | | trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 23s | | trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 31s | | branch has no errors when building and testing our client artifacts. |
| -0 :warning: | patch | 19m 55s | | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 20m 45s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 45s | | the patch passed |
| +1 :green_heart: | compile | 18m 45s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 3m 31s | | root: The patch generated 0 new + 35 unchanged - 1 fixed = 35 total (was 36) |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 26s | | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 17m 29s | | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 44m 47s | | hadoop-distcp in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not generate ASF License warnings. |
| | | 250m 21s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/3071 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8f6007b7f7af 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dbce5a20beb5c9160c44570b59ed6cfbf40772ec |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/testReport/ |
| Max. process+thread count | 2181 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-distcp U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3071/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org