You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/01/21 18:49:12 UTC
[GitHub] [hadoop] steveloughran opened a new pull request #1814:
HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
steveloughran opened a new pull request #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814
Currently AWS S3 throttling is initially handled in the AWS SDK, only reaching the S3 client code after it has given up.
This means we don't always directly observe when throttling is taking place.
Proposed:
* disable throttling retries in the AWS client Library
* add a quantile for the S3 throttle events, as DDB has
* isolate counters of s3 and DDB throttle events to classify issues better
Because we are taking over the AWS retries, we will need to expand the initial delay en retries and the number of retries we should support before giving up.
Also: should we log throttling events? It could be useful but there is a risk of logs overloading especially if many threads in the same process were triggering the problem.
Change-Id: I386928cd478a6a9fbb91f15b9185a1ea91878680
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-580692164
oops, accidentally closed this. Will resubmit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579910544
My "little" fix to turn off retries in the AWS client causes issues in the DDB clients where there's a significant mismatch between prepaid IO and load; ITestDynamoDBMetadataStoreScale is the example of this.
Looking at the AWS metrics, part of the fun is that the way bursty traffic is handled, you may get your capacity at the time of the initial load, but get blocked after. That is: the throttling may not happen under load, but during the next time a low-load API call is made.
Also, S3GuardTableAccess isn't retrying, and some code in tests and the purge/dump table entry points go on to fail when throttling happens when iterating through scans. Fix: you can ask a DDBMetastore to wrap your scan with one bonded to its retry and metrics...plus use of this where appropriate.
ITestDynamoDBMetadataStoreScale is really slow; either the changes make it worse, or its always been really slow and we haven't noticed as it was happening during the (slow) parallel test runs. Proposed: we review it, look at what we want to show and then see if we can make things fail faster
Latest Patch makes the SDK throttling disablement exclusive to S3, fixed up DDB clients to retry better and tries to make a better case for that ITestDynamoDBMetadataStoreScale suite.
I think I'm going to tune those tests to always downgrade if none is detected.
```
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale
[ERROR] Tests run: 11, Failures: 5, Errors: 1, Skipped: 0, Time elapsed: 190.404 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale
[ERROR] test_030_BatchedWrite(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 10.259 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write throttles = 0; batch throttles = 0; scan throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:578)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_030_BatchedWrite(ITestDynamoDBMetadataStoreScale.java:285)
[ERROR] test_040_get(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 4.15 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write throttles = 0; batch throttles = 0; scan throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:578)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_040_get(ITestDynamoDBMetadataStoreScale.java:341)
[ERROR] test_050_getVersionMarkerItem(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 3.311 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write throttles = 0; batch throttles = 0; scan throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:578)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_050_getVersionMarkerItem(ITestDynamoDBMetadataStoreScale.java:356)
[ERROR] test_070_putDirMarker(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 2.486 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSServiceThrottledException: getVersionMarkerItem on ../VERSION: com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: 52JGLGQ7B8SLQD3BDQCI9U6NH3VV4KQNSO5AEMVJF66Q9ASUAAJG): The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: 52JGLGQ7B8SLQD3BDQCI9U6NH3VV4KQNSO5AEMVJF66Q9ASUAAJG)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.createMetadataStore(ITestDynamoDBMetadataStoreScale.java:153)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.setup(ITestDynamoDBMetadataStoreScale.java:163)
Caused by: com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: 52JGLGQ7B8SLQD3BDQCI9U6NH3VV4KQNSO5AEMVJF66Q9ASUAAJG)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.createMetadataStore(ITestDynamoDBMetadataStoreScale.java:153)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.setup(ITestDynamoDBMetadataStoreScale.java:163)
[ERROR] test_090_delete(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 2.804 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write throttles = 0; batch throttles = 0; scan throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:578)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_090_delete(ITestDynamoDBMetadataStoreScale.java:462)
[ERROR] test_100_forgetMetadata(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 2.278 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write throttles = 0; batch throttles = 0; scan throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:578)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_100_forgetMetadata(ITestDynamoDBMetadataStoreScale.java:478)
```
For the setup failure (here in test_070_putDirMarker); not sure. We either skip the test or retry.
It's always surfacing in test_070; test_060 tests list scale. Looking at that code, I think the retry logic is too coarse -it retries the entire list, when we may want to just retry on the hasnext/next calls. That is: push it down. This will avoid so much load on any retry.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579838332
Did manage to overload one of the DDB scale tests here -those throttling values are clearly too low to recover from a big mismatch of DDB capacity and load. We are going to need some bigger values
```
ERROR] test_060_list(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 151.579 s <<< ERROR!
com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: TA7TTFE2BIFUHAH175T0GTEU6VVV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:4279)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:4246)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeScan(AmazonDynamoDBClient.java:3040)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.scan(AmazonDynamoDBClient.java:3006)
at com.amazonaws.services.dynamodbv2.document.internal.ScanCollection.firstPage(ScanCollection.java:53)
at com.amazonaws.services.dynamodbv2.document.internal.PageIterator.next(PageIterator.java:45)
at com.amazonaws.services.dynamodbv2.document.internal.IteratorSupport.nextResource(IteratorSupport.java:87)
at com.amazonaws.services.dynamodbv2.document.internal.IteratorSupport.hasNext(IteratorSupport.java:55)
at org.apache.hadoop.fs.s3a.s3guard.S3GuardTableAccess$DDBPathMetadataIterator.hasNext(S3GuardTableAccess.java:195)
at java.util.Iterator.forEachRemaining(Iterator.java:115)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.teardown(ITestDynamoDBMetadataStoreScale.java:188)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1814:
HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-576887564
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 7s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 39s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 35s | trunk passed |
| +1 :green_heart: | compile | 22m 27s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 50s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 52s | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 19s | trunk passed |
| +0 :ok: | spotbugs | 1m 19s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 45s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 35s | the patch passed |
| +1 :green_heart: | compile | 20m 25s | the patch passed |
| +1 :green_heart: | javac | 20m 25s | the patch passed |
| -0 :warning: | checkstyle | 3m 11s | root: The patch generated 1 new + 36 unchanged - 1 fixed = 37 total (was 37) |
| +1 :green_heart: | mvnsite | 2m 20s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 15m 23s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 16s | the patch passed |
| +1 :green_heart: | findbugs | 4m 5s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 42s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 55s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | The patch does not generate ASF License warnings. |
| | | 150m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 1304b5a27d2d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / f206b73 |
| Default Java | 1.8.0_232 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/artifact/out/diff-checkstyle-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/testReport/ |
| Max. process+thread count | 1344 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-577150476
might make sense to add throttling as part of the fault injection to the unreliable S3 client; we could test the counters and aid production testing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1814:
HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579894182
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 5s | trunk passed |
| +1 :green_heart: | compile | 17m 5s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 48s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 21s | trunk passed |
| +1 :green_heart: | shadedclient | 19m 12s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 15s | trunk passed |
| +0 :ok: | spotbugs | 1m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 14s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 19s | the patch passed |
| +1 :green_heart: | compile | 15m 53s | the patch passed |
| +1 :green_heart: | javac | 15m 53s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 33s | root: The patch generated 0 new + 36 unchanged - 1 fixed = 36 total (was 37) |
| +1 :green_heart: | mvnsite | 2m 15s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 51s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 18s | the patch passed |
| +1 :green_heart: | findbugs | 4m 1s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 48s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 34s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. |
| | | 123m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 209986800d88 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 7f3e1e0 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/testReport/ |
| Max. process+thread count | 1606 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579956366
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 18m 15s | trunk passed |
| +1 :green_heart: | compile | 16m 50s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 41s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 16s | trunk passed |
| +1 :green_heart: | shadedclient | 18m 49s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 14s | trunk passed |
| +0 :ok: | spotbugs | 1m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 13s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 18s | the patch passed |
| +1 :green_heart: | compile | 15m 50s | the patch passed |
| +1 :green_heart: | javac | 15m 50s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 39s | root: The patch generated 0 new + 67 unchanged - 2 fixed = 67 total (was 69) |
| +1 :green_heart: | mvnsite | 2m 16s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 12m 25s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 15s | the patch passed |
| +1 :green_heart: | findbugs | 3m 30s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 15s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 36s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. |
| | | 118m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 9eff10e7ef6c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 799d4c1 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/testReport/ |
| Max. process+thread count | 1655 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-576824665
*not yet tested against anything*
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1814:
HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579956366
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 18m 15s | trunk passed |
| +1 :green_heart: | compile | 16m 50s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 41s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 16s | trunk passed |
| +1 :green_heart: | shadedclient | 18m 49s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 14s | trunk passed |
| +0 :ok: | spotbugs | 1m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 13s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 18s | the patch passed |
| +1 :green_heart: | compile | 15m 50s | the patch passed |
| +1 :green_heart: | javac | 15m 50s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 39s | root: The patch generated 0 new + 67 unchanged - 2 fixed = 67 total (was 69) |
| +1 :green_heart: | mvnsite | 2m 16s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 12m 25s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 15s | the patch passed |
| +1 :green_heart: | findbugs | 3m 30s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 15s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 36s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 54s | The patch does not generate ASF License warnings. |
| | | 118m 8s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 9eff10e7ef6c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 799d4c1 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/testReport/ |
| Max. process+thread count | 1655 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/3/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579838804
...oh, and note that failure was in the SCAN operation. I'm not sure we have enough wrapping there
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579894182
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 5s | trunk passed |
| +1 :green_heart: | compile | 17m 5s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 48s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 21s | trunk passed |
| +1 :green_heart: | shadedclient | 19m 12s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 15s | trunk passed |
| +0 :ok: | spotbugs | 1m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 14s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 19s | the patch passed |
| +1 :green_heart: | compile | 15m 53s | the patch passed |
| +1 :green_heart: | javac | 15m 53s | the patch passed |
| +1 :green_heart: | checkstyle | 2m 33s | root: The patch generated 0 new + 36 unchanged - 1 fixed = 36 total (was 37) |
| +1 :green_heart: | mvnsite | 2m 15s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 51s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 18s | the patch passed |
| +1 :green_heart: | findbugs | 4m 1s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 9m 48s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 34s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. |
| | | 123m 42s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 209986800d88 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 7f3e1e0 |
| Default Java | 1.8.0_232 |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/testReport/ |
| Max. process+thread count | 1606 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/2/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran closed pull request #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-580001843
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 43s | trunk passed |
| +1 :green_heart: | compile | 18m 37s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 59s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | trunk passed |
| +1 :green_heart: | shadedclient | 21m 44s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 12s | trunk passed |
| +0 :ok: | spotbugs | 1m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 26s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 27s | the patch passed |
| +1 :green_heart: | compile | 16m 35s | the patch passed |
| +1 :green_heart: | javac | 16m 35s | the patch passed |
| -0 :warning: | checkstyle | 2m 53s | root: The patch generated 2 new + 66 unchanged - 3 fixed = 68 total (was 69) |
| +1 :green_heart: | mvnsite | 2m 28s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 53s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 23s | the patch passed |
| +1 :green_heart: | findbugs | 3m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 15s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 42s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 55s | The patch does not generate ASF License warnings. |
| | | 132m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 8678ec34ca80 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 799d4c1 |
| Default Java | 1.8.0_232 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/artifact/out/diff-checkstyle-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/testReport/ |
| Max. process+thread count | 1604 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-578725556
style
```
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Statistic.java:233: STORE_IO_THROTTLE_RATE("store_io_throttle_rate", "Rate of S3 request throttling"),: Line is longer than 80 characters (found 84). [LineLength]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-580249857
Quick followup: noticed there's no javadocs here. I know not all the existing constants have them, but they should.
Not worth fixing now -but for future patches, can you add a javadoc with the {@value} reference -so that anyone looking at the javadocs gets to see what the constant does and what it is. thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1814:
HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus removed a comment on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-580001843
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 37s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 43s | trunk passed |
| +1 :green_heart: | compile | 18m 37s | trunk passed |
| +1 :green_heart: | checkstyle | 2m 59s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | trunk passed |
| +1 :green_heart: | shadedclient | 21m 44s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 12s | trunk passed |
| +0 :ok: | spotbugs | 1m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 26s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 27s | the patch passed |
| +1 :green_heart: | compile | 16m 35s | the patch passed |
| +1 :green_heart: | javac | 16m 35s | the patch passed |
| -0 :warning: | checkstyle | 2m 53s | root: The patch generated 2 new + 66 unchanged - 3 fixed = 68 total (was 69) |
| +1 :green_heart: | mvnsite | 2m 28s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 53s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 23s | the patch passed |
| +1 :green_heart: | findbugs | 3m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 15s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 42s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 55s | The patch does not generate ASF License warnings. |
| | | 132m 58s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 8678ec34ca80 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 799d4c1 |
| Default Java | 1.8.0_232 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/artifact/out/diff-checkstyle-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/testReport/ |
| Max. process+thread count | 1604 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/4/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-576887564
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 2m 7s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 39s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 35s | trunk passed |
| +1 :green_heart: | compile | 22m 27s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 50s | trunk passed |
| +1 :green_heart: | mvnsite | 2m 52s | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 19s | trunk passed |
| +0 :ok: | spotbugs | 1m 19s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 3m 45s | trunk passed |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 35s | the patch passed |
| +1 :green_heart: | compile | 20m 25s | the patch passed |
| +1 :green_heart: | javac | 20m 25s | the patch passed |
| -0 :warning: | checkstyle | 3m 11s | root: The patch generated 1 new + 36 unchanged - 1 fixed = 37 total (was 37) |
| +1 :green_heart: | mvnsite | 2m 20s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 15m 23s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 2m 16s | the patch passed |
| +1 :green_heart: | findbugs | 4m 5s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 10m 42s | hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 1m 55s | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | The patch does not generate ASF License warnings. |
| | | 150m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/1814 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 1304b5a27d2d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / f206b73 |
| Default Java | 1.8.0_232 |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/artifact/out/diff-checkstyle-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/testReport/ |
| Max. process+thread count | 1344 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1814/1/console |
| versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1814: HADOOP-16823.
Manage S3 Throttling exclusively in S3A client.
Posted by GitBox <gi...@apache.org>.
steveloughran commented on issue #1814: HADOOP-16823. Manage S3 Throttling exclusively in S3A client.
URL: https://github.com/apache/hadoop/pull/1814#issuecomment-579868765
With retries on the scans in the test teardown (and dump/purge DDB), getting a failure in queryVersionMarker()
```
[ERROR] test_070_putDirMarker(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 308.882 s <<< ERROR!
org.apache.hadoop.fs.s3a.AWSServiceThrottledException: getVersionMarkerItem on ../VERSION: com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: EGKO5AOIAOMKVQE8CLR1IGP8GJVV4KQNSO5AEMVJF66Q9ASUAAJG): The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: EGKO5AOIAOMKVQE8CLR1IGP8GJVV4KQNSO5AEMVJF66Q9ASUAAJG)
at org.apache.hadoop.fs.s3a.S3AUtils.translateDynamoDBException(S3AUtils.java:424)
at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:209)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:112)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:315)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:407)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:311)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:286)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.queryVersionMarker(DynamoDBMetadataStoreTableManager.java:662)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.getVersionMarkerItem(DynamoDBMetadataStoreTableManager.java:618)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.initTable(DynamoDBMetadataStoreTableManager.java:199)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:529)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.createMetadataStore(ITestDynamoDBMetadataStoreScale.java:152)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.setup(ITestDynamoDBMetadataStoreScale.java:162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.services.dynamodbv2.model.ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ProvisionedThroughputExceededException; Request ID: EGKO5AOIAOMKVQE8CLR1IGP8GJVV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:4279)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:4246)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeGetItem(AmazonDynamoDBClient.java:2054)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:2020)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.doLoadItem(GetItemImpl.java:77)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.getItemOutcome(GetItemImpl.java:46)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.getItem(GetItemImpl.java:88)
at com.amazonaws.services.dynamodbv2.document.Table.getItem(Table.java:597)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.lambda$queryVersionMarker$2(DynamoDBMetadataStoreTableManager.java:664)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:110)
... 24 more
```
What is funny is that other tests are failing because they aren't detecting throttling:
```
[ERROR] test_100_forgetMetadata(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale) Time elapsed: 2.855 s <<< FAILURE!
java.lang.AssertionError: No throttling detected in Tracker with read throttle events = 0; write events = 0; batch throttles = 0 against DynamoDBMetadataStore{region=eu-west-1, tableName=s3guard-metadata, tableArn=arn:aws:dynamodb:eu-west-1:980678866538:table/s3guard-metadata}
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.hadoop.fs.s3a.s3guard.ThrottleTracker.assertThrottlingDetected(ThrottleTracker.java:97)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.execute(ITestDynamoDBMetadataStoreScale.java:552)
at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStoreScale.test_100_forgetMetadata(ITestDynamoDBMetadataStoreScale.java:452)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org