You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/07/24 15:08:53 UTC
[GitHub] [hadoop] steveloughran opened a new pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
steveloughran opened a new pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170
This contains the getFileStatus changes of #2149
This does not change dir marker deletion, but it
* Changes s3GetFileStatus to do HEAD + LIST only
* add in StatusProbeEnum from the 404 work
* Fix relevant tests to work -tweaking the FileOperationsCost rather than
the big move to the new cost test framework
These changes can actually go in ahead of the dir marker patch as they should marginally speed up directory listings everywhere.
A variant of this patch will be needed for 3.0+ branches; this will be the basis for those.
( I should probably have a separate JIRA ID...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran edited a comment on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
steveloughran edited a comment on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663591666
failure on unguarded run
```
[ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 36.712 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[ERROR] testCostOfGetFileStatusOnNonEmptyDir(org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost) Time elapsed: 5.415 s <<< FAILURE!
java.lang.AssertionError: Count of object_metadata_requests starting=5 current=6 diff=1: object_metadata_requests expected:<2> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:691)
at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:700)
at org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir(ITestS3AFileOperationCost.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663644838
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 11m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ branch-3.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 8s | branch-3.2 passed |
| +1 :green_heart: | compile | 15m 48s | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 2m 25s | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 2m 2s | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 17m 55s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 41s | branch-3.2 passed |
| +0 :ok: | spotbugs | 1m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 0s | branch-3.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 24s | the patch passed |
| +1 :green_heart: | compile | 15m 7s | the patch passed |
| +1 :green_heart: | javac | 15m 7s | the patch passed |
| -0 :warning: | checkstyle | 2m 25s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) |
| +1 :green_heart: | mvnsite | 1m 59s | the patch passed |
| -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 13m 8s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 34s | the patch passed |
| +1 :green_heart: | findbugs | 3m 26s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 26s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 4m 38s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. |
| | | 130m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2170 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c4b6daf3e36e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.2 / 0fb7c48 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/whitespace-eol.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/testReport/ |
| Max. process+thread count | 1370 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663656386
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 12m 47s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ branch-3.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | branch-3.2 passed |
| +1 :green_heart: | compile | 15m 25s | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 2m 26s | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 2m 3s | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 19m 3s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 45s | branch-3.2 passed |
| +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 56s | branch-3.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | the patch passed |
| +1 :green_heart: | compile | 14m 35s | the patch passed |
| +1 :green_heart: | javac | 14m 35s | the patch passed |
| -0 :warning: | checkstyle | 2m 26s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) |
| +1 :green_heart: | mvnsite | 2m 4s | the patch passed |
| -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 12m 36s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 45s | the patch passed |
| +1 :green_heart: | findbugs | 3m 16s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 8m 59s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 4m 45s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. |
| | | 131m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2170 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 92f7e0490731 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.2 / 0fb7c48 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/whitespace-eol.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/testReport/ |
| Max. process+thread count | 1385 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663656386
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 12m 47s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ branch-3.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | branch-3.2 passed |
| +1 :green_heart: | compile | 15m 25s | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 2m 26s | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 2m 3s | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 19m 3s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 45s | branch-3.2 passed |
| +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 56s | branch-3.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | the patch passed |
| +1 :green_heart: | compile | 14m 35s | the patch passed |
| +1 :green_heart: | javac | 14m 35s | the patch passed |
| -0 :warning: | checkstyle | 2m 26s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) |
| +1 :green_heart: | mvnsite | 2m 4s | the patch passed |
| -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 12m 36s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 45s | the patch passed |
| +1 :green_heart: | findbugs | 3m 16s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 8m 59s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 4m 45s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. |
| | | 131m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2170 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 92f7e0490731 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.2 / 0fb7c48 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/whitespace-eol.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/testReport/ |
| Max. process+thread count | 1385 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
steveloughran closed pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663644838
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 11m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ branch-3.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 8s | branch-3.2 passed |
| +1 :green_heart: | compile | 15m 48s | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 2m 25s | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 2m 2s | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 17m 55s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 41s | branch-3.2 passed |
| +0 :ok: | spotbugs | 1m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 0s | branch-3.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 24s | the patch passed |
| +1 :green_heart: | compile | 15m 7s | the patch passed |
| +1 :green_heart: | javac | 15m 7s | the patch passed |
| -0 :warning: | checkstyle | 2m 25s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) |
| +1 :green_heart: | mvnsite | 1m 59s | the patch passed |
| -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 13m 8s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 34s | the patch passed |
| +1 :green_heart: | findbugs | 3m 26s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 26s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 4m 38s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. |
| | | 130m 7s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2170 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c4b6daf3e36e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.2 / 0fb7c48 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/whitespace-eol.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/testReport/ |
| Max. process+thread count | 1370 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663591666
```
[ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 36.712 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost
[ERROR] testCostOfGetFileStatusOnNonEmptyDir(org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost) Time elapsed: 5.415 s <<< FAILURE!
java.lang.AssertionError: Count of object_metadata_requests starting=5 current=6 diff=1: object_metadata_requests expected:<2> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:691)
at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:700)
at org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir(ITestS3AFileOperationCost.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-666958429
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 20m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. |
||| _ branch-3.2 Compile Tests _ |
| +0 :ok: | mvndep | 0m 51s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 54s | branch-3.2 passed |
| +1 :green_heart: | compile | 18m 32s | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 2m 49s | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 1m 53s | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 19m 53s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 42s | branch-3.2 passed |
| +0 :ok: | spotbugs | 1m 2s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 16s | branch-3.2 passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 31s | the patch passed |
| +1 :green_heart: | compile | 16m 16s | the patch passed |
| +1 :green_heart: | javac | 16m 16s | the patch passed |
| -0 :warning: | checkstyle | 2m 46s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) |
| +1 :green_heart: | mvnsite | 2m 3s | the patch passed |
| -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | shadedclient | 14m 6s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 34s | the patch passed |
| +1 :green_heart: | findbugs | 3m 21s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 55s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 4m 26s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | The patch does not generate ASF License warnings. |
| | | 147m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2170 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c7ef8f03d436 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.2 / 863689f |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/whitespace-eol.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2170/2/testReport/ |
| Max. process+thread count | 1378 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2170/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2170:
URL: https://github.com/apache/hadoop/pull/2170#issuecomment-664218542
```
[ERROR] testCreateSubdirWithDifferentKey(org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC) Time elapsed: 1.54 s <<< FAILURE!
java.lang.AssertionError: Expected a java.nio.file.AccessDeniedException to be thrown, but got the result: : S3AFileStatus{path=s3a://stevel-ireland/test/testCreateSubdirWithDifferentKey/nestedDir; isDirectory=true; modification_time=0; access_time=0; owner=stevel; group=stevel; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE
at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:492)
at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:377)
at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:446)
at org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEC.testCreateSubdirWithDifferentKey(ITestS3AEncryptionSSEC.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale
[WARNING] Tests run: 9, Failures: 0, Errors: 0, Skipped: 9, Time elapsed: 4.045 s - in org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] ITestS3AEncryptionSSEC.testCreateSubdirWithDifferentKey:124 Expected a java.nio.file.AccessDeniedException to be thrown, but got the result: : S3AFileStatus{path=s3a://stevel-ireland/test/testCreateSubdirWithDifferentKey/nestedDir; isDirectory=true; modification_time=0; access_time=0; owner=stevel; group=stevel; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=FALSE
[ERROR] ITestS3AEncryptionSSEC.testListEncryptedDir:208 Expecting java.nio.file.AccessDeniedException with text Service: Amazon S3; Status Code: 403; but got : void
[ERROR] ITestS3AEncryptionSSEC.testListStatusEncryptedDir:253 Expecting java.nio.file.AccessDeniedException with text Service: Amazon S3; Status Code: 403; but got : void
[INFO]
[ERROR] Tests run: 80, Failures: 3, Errors: 0, Skipped: 9
[INFO]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org