You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "George Ladutsko (Jira)" <ji...@apache.org> on 2022/07/27 08:19:00 UTC
[jira] [Commented] (CAMEL-16287) camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create
[ https://issues.apache.org/jira/browse/CAMEL-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17571793#comment-17571793 ]
George Ladutsko commented on CAMEL-16287:
-----------------------------------------
Unfortunately, after fix (3.14.4) an application throws an exception:
{panel:title=Exception}
Caused by: software.amazon.awssdk.services.sqs.model.SqsException: MaxResults is a mandatory parameter when you provide a value for NextToken. (Service: Sqs, Status Code: 400, Request ID: 8653ea4e-3dfb-5ea6-89fc-b114d624928d)
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:167) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45) ~[sdk-core-2.17.174.jar:na]
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56) ~[aws-core-2.17.174.jar:na]
at software.amazon.awssdk.services.sqs.DefaultSqsClient.listQueues(DefaultSqsClient.java:1126) ~[sqs-2.17.99.jar:na]
at org.apache.camel.component.aws2.sqs.Sqs2Endpoint.doInit(Sqs2Endpoint.java:185) ~[camel-aws2-sqs-3.14.4.jar:3.14.4] *<=== THE PROBLEM IS HERE*
at org.apache.camel.support.service.BaseService.init(BaseService.java:83) ~[camel-api-3.14.4.jar:3.14.4]
at org.apache.camel.support.service.ServiceHelper.initService(ServiceHelper.java:84) ~[camel-api-3.14.4.jar:3.14.4]
at org.apache.camel.impl.engine.AbstractCamelContext.internalAddService(AbstractCamelContext.java:1530) ~[camel-base-engine-3.14.4.jar:3.14.4]
at org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1475) ~[camel-base-engine-3.14.4.jar:3.14.4]
at org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1470) ~[camel-base-engine-3.14.4.jar:3.14.4]
at org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1465) ~[camel-base-engine-3.14.4.jar:3.14.4]
at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:952) ~[camel-base-engine-3.14.4.jar:3.14.4]
... 44 common frames omitted
{panel}
> camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create
> ---------------------------------------------------------------------------------------
>
> Key: CAMEL-16287
> URL: https://issues.apache.org/jira/browse/CAMEL-16287
> Project: Camel
> Issue Type: Bug
> Affects Versions: 2.25.3, 3.8.0
> Reporter: Gyorgy Abraham
> Assignee: Andrea Cosentino
> Priority: Major
> Fix For: 3.14.4, 3.18.0
>
>
> There is an auto create queue feature for aws-sqs in camel-sqs component. Upon camel context creation / route building, it issues an API call to list all sqs queues, so it skips the creation for already existing ones. If camel context has a for example consumer route for an sqs queue that is not listed in the response of this API call, camel will try to to create it.
> However, this API only lists the first 1000 queues in the current account. If there are more then 1000 queues, camel-sqs will try to create a new one, and might fail. In our company's account, we have 1442 queues in our account and Camel tried to create a .fifo queue because it thought it wasnt already existing, resulting in application startup error.
> Relevant code: [https://github.com/apache/camel/blob/master/components/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Endpoint.java#L174]
> Relevant API docs at AWS: [https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/AmazonSQS.html#listQueues-com.amazonaws.services.sqs.model.ListQueuesRequest-]
> Solution is straightforward: use pagination and load all queues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)