You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Andrea Cosentino (Jira)" <ji...@apache.org> on 2020/09/14 05:59:01 UTC
[jira] [Resolved] (CAMEL-15530) AWS2 S3 Component unclosed stream
issue with includeFolders property
[ https://issues.apache.org/jira/browse/CAMEL-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrea Cosentino resolved CAMEL-15530.
--------------------------------------
Resolution: Fixed
> AWS2 S3 Component unclosed stream issue with includeFolders property
> --------------------------------------------------------------------
>
> Key: CAMEL-15530
> URL: https://issues.apache.org/jira/browse/CAMEL-15530
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2
> Affects Versions: 3.5.0
> Reporter: Filipe Portes
> Assignee: Andrea Cosentino
> Priority: Major
> Labels: easyfix, pull-request-available
> Fix For: 3.4.4, 3.6.0
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When using Camel aws2-s3 component and the `includeFolders` option as false, some `ResponseInputStream` objects are left unclosed and are causing the aws sdk connection pool to get full.
> example of Route:
> {code:java}
> from("aws2-s3://{{aws.s3.sftpBucket}}?amazonS3Client=#amazonS3&delay=1000"
> + "&includeFolders=false")
> .to("file:/tmp/?fileName=${header.CamelAwsS3Key}"){code}
> this results in the exception appearing in the logs after some seconds:
> {code:java}
> Caused by: [software.amazon.awssdk.core.exception.SdkClientException - Unable to execute HTTP request: Timeout waiting for connection from pool]software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Timeout waiting for connection from pool
> at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
> at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:43)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.setLastException(RetryableStageHelper.java:199)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:66)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)
> at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
> at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
> at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
> at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
> at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
> at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
> at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189)
> at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121)
> at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147)
> at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:101)
> at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
> at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
> at software.amazon.awssdk.services.s3.DefaultS3Client.listObjects(DefaultS3Client.java:4792)
> at org.apache.camel.component.aws2.s3.AWS2S3Consumer.poll(AWS2S3Consumer.java:138)
> at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:190)
> at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:107)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:282)
> at jdk.internal.reflect.GeneratedMethodAccessor305.invoke(Unknown Source)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at software.amazon.awssdk.http.apache.internal.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:83)
> at com.sun.proxy.$Proxy137.get(Unknown Source)
> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> at software.amazon.awssdk.http.apache.internal.impl.ApacheSdkHttpClient.execute(ApacheSdkHttpClient.java:72)
> at software.amazon.awssdk.http.apache.ApacheHttpClient.execute(ApacheHttpClient.java:232)
> at software.amazon.awssdk.http.apache.ApacheHttpClient.access$500(ApacheHttpClient.java:98)
> at software.amazon.awssdk.http.apache.ApacheHttpClient$1.call(ApacheHttpClient.java:213)
> {code}
>
> The issue doesn't repeat when the `includeFolders` is kept as true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)