You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "lqjacklee (JIRA)" <ji...@apache.org> on 2018/12/10 02:20:00 UTC
[jira] [Updated] (HADOOP-15991) testMultipartUpload
[ https://issues.apache.org/jira/browse/HADOOP-15991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
lqjacklee updated HADOOP-15991:
-------------------------------
Description:
2018-12-10 09:58:56,482 [Thread-746] INFO contract.AbstractFSContractTestBase (AbstractFSContractTestBase.java:setup(184)) - Test filesystem = s3a://jack-testlambda implemented by S3AFileSystem{uri=s3a://jack-testlambda, workingDir=s3a://jack-testlambda/user/jack, inputPolicy=normal, partSize=104857600, enableMultiObjectsDelete=true, maxKeys=5000, readAhead=65536, blockSize=33554432, multiPartThreshold=2147483647, serverSideEncryptionAlgorithm='NONE', blockFactory=org.apache.hadoop.fs.s3a.S3ADataBlocks$DiskBlockFactory@3111299f, metastore=DynamoDBMetadataStore
{region=ap-southeast-1, tableName=test-h}
, authoritative=false, useListV1=false, magicCommitter=false, boundedExecutor=BlockingThreadPoolExecutorService{SemaphoredDelegatingExecutor
{permitCount=25, available=25, waiting=0}
, activeCount=0}, unboundedExecutor=java.util.concurrent.ThreadPoolExecutor@78c596aa[Running, pool size = 10, active threads = 0, queued tasks = 0, completed tasks = 20], credentials=AWSCredentialProviderList[refcount= 2: [SimpleAWSCredentialsProvider, EnvironmentVariableCredentialsProvider, com.amazonaws.auth.InstanceProfileCredentialsProvider@4a214309], statistics \{18901220 bytes read, 18912956 bytes written, 580 read ops, 0 large read ops, 843 write ops}, metrics {
{Context=s3aFileSystem}
{s3aFileSystemId=94fd09ed-8145-4d1e-b11b-678151785e0b} \{bucket=jack-testlambda} \{stream_opened=28} \{stream_close_operations=28} \{stream_closed=28} \{stream_aborted=0} \{stream_seek_operations=0} \{stream_read_exceptions=0} \{stream_forward_seek_operations=0} \{stream_backward_seek_operations=0} \{stream_bytes_skipped_on_seek=0} \{stream_bytes_backwards_on_seek=0} \{stream_bytes_read=18901220} \{stream_read_operations=2734} \{stream_read_fully_operations=0} \{stream_read_operations_incomplete=2171} \{stream_bytes_read_in_close=0} \{stream_bytes_discarded_in_abort=0} \{files_created=49} \{files_copied=20} \{files_copied_bytes=9441684} \{files_deleted=79} \{fake_directories_deleted=630} \{directories_created=98} \{directories_deleted=19} \{ignored_errors=71} \{op_copy_from_local_file=0} \{op_create=55} \{op_create_non_recursive=6} \{op_delete=74} \{op_exists=80} \{op_get_file_checksum=8} \{op_get_file_status=923} \{op_glob_status=19} \{op_is_directory=0} \{op_is_file=0} \{op_list_files=8} \{op_list_located_status=0} \{op_list_status=51} \{op_mkdirs=114} \{op_open=28} \{op_rename=20} \{object_copy_requests=0} \{object_delete_requests=198} \{object_list_requests=337} \{object_continue_list_requests=0} \{object_metadata_requests=560} \{object_multipart_aborted=0} \{object_put_bytes=18904764} \{object_put_requests=147} \{object_put_requests_completed=147} \{stream_write_failures=0} \{stream_write_block_uploads=0} \{stream_write_block_uploads_committed=0} \{stream_write_block_uploads_aborted=0} \{stream_write_total_time=0} \{stream_write_total_data=18904764} \{committer_commits_created=0} \{committer_commits_completed=0} \{committer_jobs_completed=0} \{committer_jobs_failed=0} \{committer_tasks_completed=0} \{committer_tasks_failed=0} \{committer_bytes_committed=0} \{committer_bytes_uploaded=0} \{committer_commits_failed=0} \{committer_commits_aborted=0} \{committer_commits_reverted=0} \{committer_magic_files_created=0} \{s3guard_metadatastore_put_path_request=166} \{s3guard_metadatastore_initialization=1} \{s3guard_metadatastore_retry=0} \{s3guard_metadatastore_throttled=0} \{store_io_throttled=0} \{object_put_requests_active=0} \{object_put_bytes_pending=0} \{stream_write_block_uploads_active=0} \{stream_write_block_uploads_pending=49} \{stream_write_block_uploads_data_pending=0} \{S3guard_metadatastore_put_path_latencyNumOps=1} \{S3guard_metadatastore_put_path_latency50thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency75thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency90thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency95thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency99thPercentileLatency=427507288} \{S3guard_metadatastore_throttle_rateNumEvents=0} \{S3guard_metadatastore_throttle_rate50thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate75thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate90thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate95thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate99thPercentileFrequency (Hz)=0} }}
2018-12-10 10:01:50,127 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(108)) - Teardown: aborting outstanding uploads under s3a://jack-testlambda/test
2018-12-10 10:01:50,618 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(110)) - Found 0 incomplete uploads
java.lang.Exception: test timed out after 180000 milliseconds
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at com.amazonaws.thirdparty.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at com.amazonaws.thirdparty.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at com.amazonaws.thirdparty.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:82)
at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at com.amazonaws.thirdparty.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at com.amazonaws.thirdparty.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:3443)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3419)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeGetItem(AmazonDynamoDBClient.java:1787)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:1762)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.doLoadItem(GetItemImpl.java:77)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.getItem(GetItemImpl.java:66)
at com.amazonaws.services.dynamodbv2.document.Table.getItem(Table.java:608)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$getConsistentItem$2(DynamoDBMetadataStore.java:525)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$22/1870510662.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.getConsistentItem(DynamoDBMetadataStore.java:522)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.innerGet(DynamoDBMetadataStore.java:560)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:540)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:531)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$listChildren$4(DynamoDBMetadataStore.java:627)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$38/2053745802.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.listChildren(DynamoDBMetadataStore.java:611)
at org.apache.hadoop.fs.s3a.s3guard.DescendantsIterator.next(DescendantsIterator.java:132)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.deleteSubtree(DynamoDBMetadataStore.java:505)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerDelete(S3AFileSystem.java:1802)
at org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:1711)
at org.apache.hadoop.fs.contract.ContractTestUtils.rm(ContractTestUtils.java:402)
at org.apache.hadoop.fs.contract.ContractTestUtils.cleanup(ContractTestUtils.java:378)
at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.deleteTestDirInTeardown(AbstractFSContractTestBase.java:213)
at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.teardown(AbstractFSContractTestBase.java:204)
at org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader.teardown(ITestS3AContractMultipartUploader.java:114)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
was:
2018-12-10 09:58:56,482 [Thread-746] INFO contract.AbstractFSContractTestBase (AbstractFSContractTestBase.java:setup(184)) - Test filesystem = s3a://jack-testlambda implemented by S3AFileSystem\{uri=s3a://jack-testlambda, workingDir=s3a://jack-testlambda/user/liuquan, inputPolicy=normal, partSize=104857600, enableMultiObjectsDelete=true, maxKeys=5000, readAhead=65536, blockSize=33554432, multiPartThreshold=2147483647, serverSideEncryptionAlgorithm='NONE', blockFactory=org.apache.hadoop.fs.s3a.S3ADataBlocks$DiskBlockFactory@3111299f, metastore=DynamoDBMetadataStore{region=ap-southeast-1, tableName=test-h}, authoritative=false, useListV1=false, magicCommitter=false, boundedExecutor=BlockingThreadPoolExecutorService\{SemaphoredDelegatingExecutor{permitCount=25, available=25, waiting=0}, activeCount=0}, unboundedExecutor=java.util.concurrent.ThreadPoolExecutor@78c596aa[Running, pool size = 10, active threads = 0, queued tasks = 0, completed tasks = 20], credentials=AWSCredentialProviderList[refcount= 2: [SimpleAWSCredentialsProvider, EnvironmentVariableCredentialsProvider, com.amazonaws.auth.InstanceProfileCredentialsProvider@4a214309], statistics \{18901220 bytes read, 18912956 bytes written, 580 read ops, 0 large read ops, 843 write ops}, metrics \{{Context=s3aFileSystem} \{s3aFileSystemId=94fd09ed-8145-4d1e-b11b-678151785e0b} \{bucket=jack-testlambda} \{stream_opened=28} \{stream_close_operations=28} \{stream_closed=28} \{stream_aborted=0} \{stream_seek_operations=0} \{stream_read_exceptions=0} \{stream_forward_seek_operations=0} \{stream_backward_seek_operations=0} \{stream_bytes_skipped_on_seek=0} \{stream_bytes_backwards_on_seek=0} \{stream_bytes_read=18901220} \{stream_read_operations=2734} \{stream_read_fully_operations=0} \{stream_read_operations_incomplete=2171} \{stream_bytes_read_in_close=0} \{stream_bytes_discarded_in_abort=0} \{files_created=49} \{files_copied=20} \{files_copied_bytes=9441684} \{files_deleted=79} \{fake_directories_deleted=630} \{directories_created=98} \{directories_deleted=19} \{ignored_errors=71} \{op_copy_from_local_file=0} \{op_create=55} \{op_create_non_recursive=6} \{op_delete=74} \{op_exists=80} \{op_get_file_checksum=8} \{op_get_file_status=923} \{op_glob_status=19} \{op_is_directory=0} \{op_is_file=0} \{op_list_files=8} \{op_list_located_status=0} \{op_list_status=51} \{op_mkdirs=114} \{op_open=28} \{op_rename=20} \{object_copy_requests=0} \{object_delete_requests=198} \{object_list_requests=337} \{object_continue_list_requests=0} \{object_metadata_requests=560} \{object_multipart_aborted=0} \{object_put_bytes=18904764} \{object_put_requests=147} \{object_put_requests_completed=147} \{stream_write_failures=0} \{stream_write_block_uploads=0} \{stream_write_block_uploads_committed=0} \{stream_write_block_uploads_aborted=0} \{stream_write_total_time=0} \{stream_write_total_data=18904764} \{committer_commits_created=0} \{committer_commits_completed=0} \{committer_jobs_completed=0} \{committer_jobs_failed=0} \{committer_tasks_completed=0} \{committer_tasks_failed=0} \{committer_bytes_committed=0} \{committer_bytes_uploaded=0} \{committer_commits_failed=0} \{committer_commits_aborted=0} \{committer_commits_reverted=0} \{committer_magic_files_created=0} \{s3guard_metadatastore_put_path_request=166} \{s3guard_metadatastore_initialization=1} \{s3guard_metadatastore_retry=0} \{s3guard_metadatastore_throttled=0} \{store_io_throttled=0} \{object_put_requests_active=0} \{object_put_bytes_pending=0} \{stream_write_block_uploads_active=0} \{stream_write_block_uploads_pending=49} \{stream_write_block_uploads_data_pending=0} \{S3guard_metadatastore_put_path_latencyNumOps=1} \{S3guard_metadatastore_put_path_latency50thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency75thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency90thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency95thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency99thPercentileLatency=427507288} \{S3guard_metadatastore_throttle_rateNumEvents=0} \{S3guard_metadatastore_throttle_rate50thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate75thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate90thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate95thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate99thPercentileFrequency (Hz)=0} }}
2018-12-10 10:01:50,127 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(108)) - Teardown: aborting outstanding uploads under s3a://jack-testlambda/test
2018-12-10 10:01:50,618 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(110)) - Found 0 incomplete uploads
java.lang.Exception: test timed out after 180000 milliseconds
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at com.amazonaws.thirdparty.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at com.amazonaws.thirdparty.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at com.amazonaws.thirdparty.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:82)
at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at com.amazonaws.thirdparty.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at com.amazonaws.thirdparty.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:3443)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3419)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeGetItem(AmazonDynamoDBClient.java:1787)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:1762)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.doLoadItem(GetItemImpl.java:77)
at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.getItem(GetItemImpl.java:66)
at com.amazonaws.services.dynamodbv2.document.Table.getItem(Table.java:608)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$getConsistentItem$2(DynamoDBMetadataStore.java:525)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$22/1870510662.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.getConsistentItem(DynamoDBMetadataStore.java:522)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.innerGet(DynamoDBMetadataStore.java:560)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:540)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:531)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$listChildren$4(DynamoDBMetadataStore.java:627)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$38/2053745802.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.listChildren(DynamoDBMetadataStore.java:611)
at org.apache.hadoop.fs.s3a.s3guard.DescendantsIterator.next(DescendantsIterator.java:132)
at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.deleteSubtree(DynamoDBMetadataStore.java:505)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerDelete(S3AFileSystem.java:1802)
at org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:1711)
at org.apache.hadoop.fs.contract.ContractTestUtils.rm(ContractTestUtils.java:402)
at org.apache.hadoop.fs.contract.ContractTestUtils.cleanup(ContractTestUtils.java:378)
at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.deleteTestDirInTeardown(AbstractFSContractTestBase.java:213)
at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.teardown(AbstractFSContractTestBase.java:204)
at org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader.teardown(ITestS3AContractMultipartUploader.java:114)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> testMultipartUpload
> -------------------
>
> Key: HADOOP-15991
> URL: https://issues.apache.org/jira/browse/HADOOP-15991
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
> Affects Versions: 3.2.0
> Reporter: lqjacklee
> Priority: Minor
>
> 2018-12-10 09:58:56,482 [Thread-746] INFO contract.AbstractFSContractTestBase (AbstractFSContractTestBase.java:setup(184)) - Test filesystem = s3a://jack-testlambda implemented by S3AFileSystem{uri=s3a://jack-testlambda, workingDir=s3a://jack-testlambda/user/jack, inputPolicy=normal, partSize=104857600, enableMultiObjectsDelete=true, maxKeys=5000, readAhead=65536, blockSize=33554432, multiPartThreshold=2147483647, serverSideEncryptionAlgorithm='NONE', blockFactory=org.apache.hadoop.fs.s3a.S3ADataBlocks$DiskBlockFactory@3111299f, metastore=DynamoDBMetadataStore
> {region=ap-southeast-1, tableName=test-h}
> , authoritative=false, useListV1=false, magicCommitter=false, boundedExecutor=BlockingThreadPoolExecutorService{SemaphoredDelegatingExecutor
> {permitCount=25, available=25, waiting=0}
> , activeCount=0}, unboundedExecutor=java.util.concurrent.ThreadPoolExecutor@78c596aa[Running, pool size = 10, active threads = 0, queued tasks = 0, completed tasks = 20], credentials=AWSCredentialProviderList[refcount= 2: [SimpleAWSCredentialsProvider, EnvironmentVariableCredentialsProvider, com.amazonaws.auth.InstanceProfileCredentialsProvider@4a214309], statistics \{18901220 bytes read, 18912956 bytes written, 580 read ops, 0 large read ops, 843 write ops}, metrics {
> {Context=s3aFileSystem}
> {s3aFileSystemId=94fd09ed-8145-4d1e-b11b-678151785e0b} \{bucket=jack-testlambda} \{stream_opened=28} \{stream_close_operations=28} \{stream_closed=28} \{stream_aborted=0} \{stream_seek_operations=0} \{stream_read_exceptions=0} \{stream_forward_seek_operations=0} \{stream_backward_seek_operations=0} \{stream_bytes_skipped_on_seek=0} \{stream_bytes_backwards_on_seek=0} \{stream_bytes_read=18901220} \{stream_read_operations=2734} \{stream_read_fully_operations=0} \{stream_read_operations_incomplete=2171} \{stream_bytes_read_in_close=0} \{stream_bytes_discarded_in_abort=0} \{files_created=49} \{files_copied=20} \{files_copied_bytes=9441684} \{files_deleted=79} \{fake_directories_deleted=630} \{directories_created=98} \{directories_deleted=19} \{ignored_errors=71} \{op_copy_from_local_file=0} \{op_create=55} \{op_create_non_recursive=6} \{op_delete=74} \{op_exists=80} \{op_get_file_checksum=8} \{op_get_file_status=923} \{op_glob_status=19} \{op_is_directory=0} \{op_is_file=0} \{op_list_files=8} \{op_list_located_status=0} \{op_list_status=51} \{op_mkdirs=114} \{op_open=28} \{op_rename=20} \{object_copy_requests=0} \{object_delete_requests=198} \{object_list_requests=337} \{object_continue_list_requests=0} \{object_metadata_requests=560} \{object_multipart_aborted=0} \{object_put_bytes=18904764} \{object_put_requests=147} \{object_put_requests_completed=147} \{stream_write_failures=0} \{stream_write_block_uploads=0} \{stream_write_block_uploads_committed=0} \{stream_write_block_uploads_aborted=0} \{stream_write_total_time=0} \{stream_write_total_data=18904764} \{committer_commits_created=0} \{committer_commits_completed=0} \{committer_jobs_completed=0} \{committer_jobs_failed=0} \{committer_tasks_completed=0} \{committer_tasks_failed=0} \{committer_bytes_committed=0} \{committer_bytes_uploaded=0} \{committer_commits_failed=0} \{committer_commits_aborted=0} \{committer_commits_reverted=0} \{committer_magic_files_created=0} \{s3guard_metadatastore_put_path_request=166} \{s3guard_metadatastore_initialization=1} \{s3guard_metadatastore_retry=0} \{s3guard_metadatastore_throttled=0} \{store_io_throttled=0} \{object_put_requests_active=0} \{object_put_bytes_pending=0} \{stream_write_block_uploads_active=0} \{stream_write_block_uploads_pending=49} \{stream_write_block_uploads_data_pending=0} \{S3guard_metadatastore_put_path_latencyNumOps=1} \{S3guard_metadatastore_put_path_latency50thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency75thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency90thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency95thPercentileLatency=427507288} \{S3guard_metadatastore_put_path_latency99thPercentileLatency=427507288} \{S3guard_metadatastore_throttle_rateNumEvents=0} \{S3guard_metadatastore_throttle_rate50thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate75thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate90thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate95thPercentileFrequency (Hz)=0} \{S3guard_metadatastore_throttle_rate99thPercentileFrequency (Hz)=0} }}
> 2018-12-10 10:01:50,127 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(108)) - Teardown: aborting outstanding uploads under s3a://jack-testlambda/test
> 2018-12-10 10:01:50,618 [Thread-746] INFO s3a.ITestS3AContractMultipartUploader (ITestS3AContractMultipartUploader.java:teardown(110)) - Found 0 incomplete uploads
> java.lang.Exception: test timed out after 180000 milliseconds
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
> at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
> at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
> at com.amazonaws.thirdparty.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
> at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
> at com.amazonaws.thirdparty.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> at com.amazonaws.thirdparty.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> at com.amazonaws.thirdparty.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> at com.amazonaws.thirdparty.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:82)
> at com.amazonaws.thirdparty.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> at com.amazonaws.thirdparty.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> at com.amazonaws.thirdparty.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> at com.amazonaws.thirdparty.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
> at com.amazonaws.thirdparty.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1238)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
> at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
> at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:3443)
> at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3419)
> at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeGetItem(AmazonDynamoDBClient.java:1787)
> at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:1762)
> at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.doLoadItem(GetItemImpl.java:77)
> at com.amazonaws.services.dynamodbv2.document.internal.GetItemImpl.getItem(GetItemImpl.java:66)
> at com.amazonaws.services.dynamodbv2.document.Table.getItem(Table.java:608)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$getConsistentItem$2(DynamoDBMetadataStore.java:525)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$22/1870510662.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.getConsistentItem(DynamoDBMetadataStore.java:522)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.innerGet(DynamoDBMetadataStore.java:560)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:540)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:531)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.lambda$listChildren$4(DynamoDBMetadataStore.java:627)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore$$Lambda$38/2053745802.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
> at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265)
> at org.apache.hadoop.fs.s3a.Invoker$$Lambda$16/750351714.execute(Unknown Source)
> at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:322)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:261)
> at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:236)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.listChildren(DynamoDBMetadataStore.java:611)
> at org.apache.hadoop.fs.s3a.s3guard.DescendantsIterator.next(DescendantsIterator.java:132)
> at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.deleteSubtree(DynamoDBMetadataStore.java:505)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.innerDelete(S3AFileSystem.java:1802)
> at org.apache.hadoop.fs.s3a.S3AFileSystem.delete(S3AFileSystem.java:1711)
> at org.apache.hadoop.fs.contract.ContractTestUtils.rm(ContractTestUtils.java:402)
> at org.apache.hadoop.fs.contract.ContractTestUtils.cleanup(ContractTestUtils.java:378)
> at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.deleteTestDirInTeardown(AbstractFSContractTestBase.java:213)
> at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.teardown(AbstractFSContractTestBase.java:204)
> at org.apache.hadoop.fs.contract.s3a.ITestS3AContractMultipartUploader.teardown(ITestS3AContractMultipartUploader.java:114)
> at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org