You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/12/31 12:20:55 UTC
[GitHub] [hadoop] steveloughran opened a new pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
steveloughran opened a new pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580
This is a major rework of the S3A Statistics collection to
* Move from direct references of S3AInstrumention statistics
collectors to interface/implementation classes in new packages.
* Ubiquitous support of IOStatistics, including:
S3AFileSystem, input and output streams, RemoteIterator instances
provided in list calls.
* Adoption of new statistic names from hadoop-common
Regarding statistic collection, as well as all existing
statistics, the connector now records min/max/mean durations
of HTTP GET and HEAD requests, and those of LIST operations.
Contributed by Steve Loughran.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-752944899
* This is the S3A part of #2577 (including the MR patch pending its merge)
* already approved by gabor
* rerunning the tests before the final merge
Gabor has reported a test failure but I can't replicate it. I'm going to merge this in and then add a followup stabilisation patch for that and any other inevitable surprises -no CP to branch-3.3 until that is in
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-752991931
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 25m 6s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch appears to include 45 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 34s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 11s | | trunk passed |
| +1 :green_heart: | compile | 20m 15s | | trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | compile | 17m 33s | | trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| +1 :green_heart: | checkstyle | 2m 38s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 23s | | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| +0 :ok: | spotbugs | 1m 14s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 38s | | trunk passed |
|||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 20m 40s | | the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| +1 :green_heart: | javac | 20m 40s | | root-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 generated 0 new + 2040 unchanged - 21 fixed = 2040 total (was 2061) |
| +1 :green_heart: | compile | 19m 51s | | the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| +1 :green_heart: | javac | 19m 51s | | root-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 0 new + 1935 unchanged - 21 fixed = 1935 total (was 1956) |
| -0 :warning: | checkstyle | 2m 40s | [/diff-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/artifact/out/diff-checkstyle-root.txt) | root: The patch generated 5 new + 130 unchanged - 22 fixed = 135 total (was 152) |
| +1 :green_heart: | mvnsite | 1m 47s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 15m 30s | | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 14s | | the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 |
| -1 :x: | javadoc | 0m 44s | [/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/artifact/out/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 generated 1 new + 88 unchanged - 0 fixed = 89 total (was 88) |
| -1 :x: | findbugs | 1m 25s | [/new-findbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/artifact/out/new-findbugs-hadoop-tools_hadoop-aws.html) | hadoop-tools/hadoop-aws generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 7m 11s | | hadoop-mapreduce-client-core in the patch passed. |
| +1 :green_heart: | unit | 1m 43s | | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 55s | | The patch does not generate ASF License warnings. |
| | | 205m 38s | | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-tools/hadoop-aws |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.getChangeTrackerStatistics() At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.getChangeTrackerStatistics() At S3AInstrumentation.java:[line 1056] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.getInputPolicy() At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.getInputPolicy() At S3AInstrumentation.java:[line 1265] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.inputPolicySet(int) At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.inputPolicySet(int) At S3AInstrumentation.java:[line 1045] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.merge(boolean) At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.merge(boolean) At S3AInstrumentation.java:[line 1111] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.promoteInputStreamCountersToMetrics() At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.promoteInputStreamCountersToMetrics() At S3AInstrumentation.java:[line 1150] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.toString() At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$InputStreamStatistics.toString() At S3AInstrumentation.java:[line 1072] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$OutputStreamStatistics.blockUploadStarted(Duration, int) At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$OutputStreamStatistics.blockUploadStarted(Duration, int) At S3AInstrumentation.java:[line 1410] |
| | Potentially ambiguous invocation of either an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$OutputStreamStatistics.toString() At S3AInstrumentation.java:an outer or inherited method org.apache.hadoop.fs.s3a.statistics.impl.AbstractS3AStatisticsSource.getIOStatistics() in org.apache.hadoop.fs.s3a.S3AInstrumentation$OutputStreamStatistics.toString() At S3AInstrumentation.java:[line 1556] |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2580 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 5cdb0fd7a5fd 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 99d08a19ba5 |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/testReport/ |
| Max. process+thread count | 1557 (vs. ulimit of 5500) |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-753214369
Again, this is yetus playing up; local builds and tests are fine. I am seeing an intermittent test failure on parallel runs of ITestHugeFile uploads, with a counter not incrementing, but this doesn't surface standalone. Will file and fix separately
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-753021806
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch appears to include 45 new or modified test files. |
|||| _ trunk Compile Tests _ |
| -1 :x: | mvninstall | 15m 36s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-mvninstall-root.txt) | root in trunk failed. |
| -1 :x: | compile | 0m 25s | [/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | compile | 0m 25s | [/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -0 :warning: | checkstyle | 0m 23s | [/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 51s | [/branch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. |
| +1 :green_heart: | shadedclient | 1m 47s | | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 26s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javadoc | 0m 26s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| +0 :ok: | spotbugs | 3m 7s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 0m 26s | [/branch-findbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/branch-findbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 23s | [/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| -1 :x: | compile | 0m 23s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javac | 0m 23s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | compile | 0m 23s | [/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -1 :x: | javac | 0m 23s | [/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -0 :warning: | checkstyle | 0m 20s | [/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 23s | [/patch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 0m 25s | | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 23s | [/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javadoc | 0m 23s | [/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -1 :x: | findbugs | 0m 24s | [/patch-findbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 23s | [/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +0 :ok: | asflicense | 0m 24s | | ASF License check generated no output? |
| | | 28m 31s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2580 |
| Optional Tests | dupname asflicense xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8f3231479ebe 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9b2956e254d |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/testReport/ |
| Max. process+thread count | 94 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/2/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-753010028
latest patches fix up the style/findbugs warnings.
I've also replicated and fixed the test failure gabor reported. I am seeing a failure on huge uploads and putbyte counts, but I want to do that later just to get this one in by the end of 2020.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580#issuecomment-753025232
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 32m 40s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch appears to include 45 new or modified test files. |
|||| _ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 23s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-mvninstall-root.txt) | root in trunk failed. |
| -1 :x: | compile | 0m 23s | [/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | compile | 0m 23s | [/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -0 :warning: | checkstyle | 0m 23s | [/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/buildtool-branch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 25s | [/branch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. |
| +1 :green_heart: | shadedclient | 1m 16s | | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 23s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in trunk failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javadoc | 0m 23s | [/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in trunk failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| +0 :ok: | spotbugs | 2m 29s | | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 0m 24s | [/branch-findbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/branch-findbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in trunk failed. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 24s | [/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| -1 :x: | compile | 0m 25s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javac | 0m 25s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | compile | 0m 25s | [/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -1 :x: | javac | 0m 25s | [/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -0 :warning: | checkstyle | 0m 22s | [/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/buildtool-patch-checkstyle-hadoop-tools_hadoop-aws.txt) | The patch fails to run checkstyle in hadoop-aws |
| -1 :x: | mvnsite | 0m 25s | [/patch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 0m 27s | | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 25s | [/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04. |
| -1 :x: | javadoc | 0m 27s | [/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01.txt) | hadoop-aws in the patch failed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01. |
| -1 :x: | findbugs | 0m 27s | [/patch-findbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 25s | [/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not generate ASF License warnings. |
| | | 45m 40s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2580 |
| Optional Tests | dupname asflicense xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 70e0d7a1a54e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9b2956e254d |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/testReport/ |
| Max. process+thread count | 51 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2580/3/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #2580: HADOOP-17271. S3A connector to support IOStatistics.
Posted by GitBox <gi...@apache.org>.
steveloughran merged pull request #2580:
URL: https://github.com/apache/hadoop/pull/2580
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org