You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/07/16 12:32:14 UTC
[GitHub] [hadoop] mukund-thakur opened a new pull request #2148: HADOOP-17131 Moving listing to use operation callback
mukund-thakur opened a new pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148
Ran tests in ap-south-1 bucket using mvn clean verify -Dparallel-tests and mvn clean verify -Ds3guard -Ddynamo -Dparallel-tests
All good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2148: HADOOP-17131. Refactor S3A Listing code for better isolation.
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-669095775
I was ignoring java doc issues because of https://issues.apache.org/jira/browse/HADOOP-17091. Looks like it got fixed recently. Sorry.
I will create a small patch and fix https://issues.apache.org/jira/browse/HADOOP-17186
Thanks for pointing out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2148: HADOOP-17131. Refactor S3A Listing code for better isolation.
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-669062015
Hi @steveloughran and @mukund-thakur
`mvn javadoc:javadoc` fails after this commit.
Filed https://issues.apache.org/jira/browse/HADOOP-17186
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-661094908
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | docker | 9m 51s | Docker failed to build yetus/hadoop:cce5a6f6094. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/3/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#discussion_r462484155
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OperationCallbacks.java
##########
@@ -40,7 +40,8 @@
import org.apache.hadoop.fs.s3a.s3guard.BulkOperationState;
/**
- * These are all the callbacks which the {@link RenameOperation}
+ * These are all the callbacks which the {@link RenameOperation},
Review comment:
not true any more though, not now there's a new listing interface, right?
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Listing.java
##########
@@ -46,25 +56,27 @@
import java.util.Set;
import static org.apache.hadoop.fs.s3a.Constants.S3N_FOLDER_SUFFIX;
+import static org.apache.hadoop.fs.s3a.S3AUtils.ACCEPT_ALL;
import static org.apache.hadoop.fs.s3a.S3AUtils.createFileStatus;
+import static org.apache.hadoop.fs.s3a.S3AUtils.maybeAddTrailingSlash;
import static org.apache.hadoop.fs.s3a.S3AUtils.objectRepresentsDirectory;
import static org.apache.hadoop.fs.s3a.S3AUtils.stringify;
import static org.apache.hadoop.fs.s3a.S3AUtils.translateException;
+import static org.apache.hadoop.fs.s3a.auth.RoleModel.pathToKey;
/**
* Place for the S3A listing classes; keeps all the small classes under control.
*/
@InterfaceAudience.Private
-public class Listing {
+public class Listing extends AbstractStoreOperation {
- private final S3AFileSystem owner;
private static final Logger LOG = S3AFileSystem.LOG;
static final FileStatusAcceptor ACCEPT_ALL_BUT_S3N =
new AcceptAllButS3nDirs();
- public Listing(S3AFileSystem owner) {
- this.owner = owner;
+ public Listing(StoreContext storeContext) {
Review comment:
have this take the context and the ListingOperationCallbacks, add a method in S3AFS `createListing()` which can be used there and in DumpS3GuardDynamoTable...that avoids having to extend the store context
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -4847,5 +4800,20 @@ public String getBucketLocation() throws IOException {
public Path makeQualified(final Path path) {
return S3AFileSystem.this.makeQualified(path);
}
+
Review comment:
Add to the ListingOperationCallbacks
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/StoreContext.java
##########
@@ -116,6 +116,16 @@
*/
private ITtlTimeProvider timeProvider;
+ /**
Review comment:
I'd rather these were left out of the context and only explicitly passed to those classes which explicitly needed the operations. That helps us stay in control of what operations specific modules can perform
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -1599,6 +1603,36 @@ public boolean allowAuthoritative(final Path p) {
}
}
+ protected class ListingOperationCallbacksImpl implements ListingOperationCallbacks {
+
+ @Override
+ @Retries.RetryRaw
+ public S3ListResult listObjects(
+ S3ListRequest request)
+ throws IOException {
+ return S3AFileSystem.this.listObjects(request);
+ }
+
+ @Override
+ @Retries.RetryRaw
Review comment:
nice touch
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -293,6 +294,9 @@
private final S3AFileSystem.OperationCallbacksImpl
operationCallbacks = new OperationCallbacksImpl();
+ private final S3AFileSystem.ListingOperationCallbacksImpl
Review comment:
Make the type of this field the interface for strictness
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-666938853
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 24m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 20m 45s | trunk passed |
| +1 :green_heart: | compile | 0m 42s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 24s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | trunk passed |
| +1 :green_heart: | shadedclient | 14m 46s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 31s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 3s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | the patch passed |
| +1 :green_heart: | compile | 0m 36s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 36s | the patch passed |
| +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 30s | the patch passed |
| -0 :warning: | checkstyle | 0m 20s | hadoop-tools/hadoop-aws: The patch generated 15 new + 13 unchanged - 0 fixed = 28 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 35s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 8s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 28s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 1m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 30s | hadoop-aws in the patch passed. |
| -1 :x: | asflicense | 0m 29s | The patch generated 1 ASF License warnings. |
| | | 86m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 077989e3cca9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 05b3337a460 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 452 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-668216731
yetus doesn't seem to be live right now, so we don't have a final check. But some of its earlier comments are still there by the look of things, especially
```
Lines that start with ????? in the ASF License report indicate files that do not have an Apache license header:
!????? /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2148/src/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/ListingOperationCallbacks.java
```
Can you fix that? All the checkstyles seem to have been dealt with.
+1 pending the license
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-661245111
Tested raw and guarded config in ap-south-1 bucket. All good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-660136818
Test failures:
`[ERROR] Failures:
[ERROR] ITestS3GuardOutOfBandOperations.testRootTombstones:619->Assert.assertNotEquals:161->Assert.failEquals:185->Assert.fail:88 Listing was not supposed to include s3a://mthakur-data/test/testRootTombstones[auth=false]-nonauth0da8099f-ff80-4bf1-87f4-6a1326388907/test.file. Actual: s3a://mthakur-data/test/testRootTombstones[auth=false]-nonauth0da8099f-ff80-4bf1-87f4-6a1326388907/test.file
[ERROR] ITestS3GuardOutOfBandOperations.testTombstoneExpiryGuardedDeleteRawCreate:358->Assert.assertNotEquals:161->Assert.failEquals:185->Assert.fail:88 Listing was not supposed to include s3a://mthakur-data/test/TEGDRC-13fba05d-ff3c-4a15-bb96-75942ac8f458/file. Actual: s3a://mthakur-data/test/TEGDRC-13fba05d-ff3c-4a15-bb96-75942ac8f458/file`
The reason test fails is we are mocking the ITtlTimeProvider after the fileSystem is initialised thus not propagating the updated ITtlTimeProvider instance to StoreContext.
One solution I have in my mind is to add one method to fetch the update TtlProvider in OperationCallbacks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-660181715
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 25s | trunk passed |
| +1 :green_heart: | compile | 0m 39s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 29s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 0s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 30s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 1m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 4s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | the patch passed |
| +1 :green_heart: | compile | 0m 33s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 0m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| -0 :warning: | checkstyle | 0m 20s | hadoop-tools/hadoop-aws: The patch generated 15 new + 13 unchanged - 0 fixed = 28 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 33s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 57s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 27s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 0m 23s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 1m 4s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 21s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. |
| | | 62m 38s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 59509d09749e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 2ba44a73bf2 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/testReport/ |
| Max. process+thread count | 451 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/2/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-668431613
Not fixing first checkstyle here https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/patch-asflicense-problems.txt as the line length is only 82. Cutting it will make the side by side review difficult.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#discussion_r463954321
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Listing.java
##########
@@ -68,15 +68,21 @@
* Place for the S3A listing classes; keeps all the small classes under control.
*/
@InterfaceAudience.Private
-public class Listing extends AbstractStoreOperation {
+public class Listing {
Review comment:
1. I'd keep it as a subclass of AbstractStoreOperation.
2. we may want to declare it as Closeable, and in tests and S3AFS call close()/closeQuietly() on it. Gives us the option of doing more with it later
What I don't want -and which you have done in this commit, is add the new list commands into the OperationCallbacks interface
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-666938853
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 24m 11s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 20m 45s | trunk passed |
| +1 :green_heart: | compile | 0m 42s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 24s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | trunk passed |
| +1 :green_heart: | shadedclient | 14m 46s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 31s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| +1 :green_heart: | javadoc | 0m 33s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 1m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 1m 3s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | the patch passed |
| +1 :green_heart: | compile | 0m 36s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 36s | the patch passed |
| +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 30s | the patch passed |
| -0 :warning: | checkstyle | 0m 20s | hadoop-tools/hadoop-aws: The patch generated 15 new + 13 unchanged - 0 fixed = 28 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 35s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 8s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 28s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 1m 8s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 30s | hadoop-aws in the patch passed. |
| -1 :x: | asflicense | 0m 29s | The patch generated 1 ASF License warnings. |
| | | 86m 44s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 077989e3cca9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 05b3337a460 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 452 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/3/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-659429461
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-668459991
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 3s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| -1 :x: | mvninstall | 0m 15s | root in trunk failed. |
| +1 :green_heart: | compile | 0m 34s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 28s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 18s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 30s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 27s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 0m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 55s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 33s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 26s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 26s | the patch passed |
| -0 :warning: | checkstyle | 0m 17s | hadoop-tools/hadoop-aws: The patch generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 30s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 36s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 15s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javadoc | 0m 21s | hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09 with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4) |
| +1 :green_heart: | findbugs | 1m 2s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 10s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. |
| | | 44m 18s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5d4718ffb6e4 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / ab2b3df2de1 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| mvninstall | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/artifact/out/branch-mvninstall-root.txt |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/artifact/out/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09.txt |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/testReport/ |
| Max. process+thread count | 428 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/8/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-661094908
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | docker | 9m 51s | Docker failed to build yetus/hadoop:cce5a6f6094. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/3/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #2148: HADOOP-17131. Refactor S3A Listing code for better isolation.
Posted by GitBox <gi...@apache.org>.
steveloughran merged pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-668410725
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 13s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 19m 35s | trunk passed |
| +1 :green_heart: | compile | 0m 37s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | compile | 0m 32s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 26s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | trunk passed |
| +1 :green_heart: | shadedclient | 14m 34s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 20s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javadoc | 0m 27s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 1m 0s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 58s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 35s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| +1 :green_heart: | javac | 0m 35s | the patch passed |
| +1 :green_heart: | compile | 0m 28s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 28s | the patch passed |
| -0 :warning: | checkstyle | 0m 19s | hadoop-tools/hadoop-aws: The patch generated 2 new + 13 unchanged - 0 fixed = 15 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 33s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 52s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 0m 17s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 |
| -1 :x: | javadoc | 0m 23s | hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09 with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4) |
| -1 :x: | findbugs | 1m 6s | hadoop-tools/hadoop-aws generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 29s | hadoop-aws in the patch passed. |
| -1 :x: | asflicense | 0m 29s | The patch generated 1 ASF License warnings. |
| | | 61m 25s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-tools/hadoop-aws |
| | Unused field:StoreContextBuilder.java |
| | Unused field:StoreContextBuilder.java |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 7d5540d756f6 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / ab2b3df2de1 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| checkstyle | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/diff-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_252-8u252-b09-1~18.04-b09.txt |
| findbugs | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/new-findbugs-hadoop-tools_hadoop-aws.html |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/testReport/ |
| asflicense | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 414 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2148/7/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2148: HADOOP-17131. Refactor S3A Listing code for better isolation.
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-668650769
Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on a change in pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#discussion_r462842264
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OperationCallbacks.java
##########
@@ -40,7 +40,8 @@
import org.apache.hadoop.fs.s3a.s3guard.BulkOperationState;
/**
- * These are all the callbacks which the {@link RenameOperation}
+ * These are all the callbacks which the {@link RenameOperation},
Review comment:
Ah.. I updated the doc in the start. Will fix. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#issuecomment-659429461
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 18m 58s | trunk passed |
| +1 :green_heart: | compile | 0m 41s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | compile | 0m 35s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | checkstyle | 0m 28s | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | trunk passed |
| +1 :green_heart: | shadedclient | 14m 56s | branch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 31s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 0m 30s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 0m 59s | trunk passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | the patch passed |
| +1 :green_heart: | compile | 0m 33s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 |
| +1 :green_heart: | javac | 0m 33s | the patch passed |
| +1 :green_heart: | compile | 0m 27s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | javac | 0m 27s | the patch passed |
| -0 :warning: | checkstyle | 0m 19s | hadoop-tools/hadoop-aws: The patch generated 11 new + 13 unchanged - 0 fixed = 24 total (was 13) |
| +1 :green_heart: | mvnsite | 0m 30s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 13m 50s | patch has no errors when building and testing our client artifacts. |
| -1 :x: | javadoc | 0m 28s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. |
| +1 :green_heart: | javadoc | 0m 25s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| +1 :green_heart: | findbugs | 1m 3s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 21s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. |
| | | 60m 57s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2148 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 1d386197bf5b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 1f71c4ae714 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt |
| javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/testReport/ |
| Max. process+thread count | 457 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2148/1/console |
| versions | git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a change in pull request #2148: HADOOP-17131 Moving listing to use operation callback
Posted by GitBox <gi...@apache.org>.
mukund-thakur commented on a change in pull request #2148:
URL: https://github.com/apache/hadoop/pull/2148#discussion_r462841126
##########
File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -4847,5 +4800,20 @@ public String getBucketLocation() throws IOException {
public Path makeQualified(final Path path) {
return S3AFileSystem.this.makeQualified(path);
}
+
Review comment:
Felt like below three methods are generic for context so I added here such that if required in future it can be used by other operations as well. I think the reason you are saying to add in ListingOperationCallbacks is because these are currently only used in Listing Operations.
I can put in Listing as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org