[jira] [Commented] (CAMEL-16287) camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create

["Stephen Lawson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17534324#comment-17534324 ] 

Stephen Lawson commented on CAMEL-16287:
----------------------------------------

Hi. Sorry to be a pain but the fix is not quite right (v 3.14.3) . When the nextToken is null then no further calls to the listQueues() method should be called. I think this would be better:

 
{code:java}
// check whether the queue already exists
boolean done = false;
while (!done) {
    ListQueuesResponse listQueuesResult
            = client.listQueues(ListQueuesRequest.builder().maxResults(1000).build());    for (String url : listQueuesResult.queueUrls()) {
        if (url.endsWith("/" + configuration.getQueueName())) {
            queueUrl = url;
            LOG.trace("Queue available at '{}'.", queueUrl);
            break;
        }
    }   
    if (listQueuesResult.nextToken() == null) {
        done = true;
        continue;
    }   
    String token = listQueuesResult.nextToken();
    listQueuesResult = client.listQueues(ListQueuesRequest.builder().nextToken(token).build());
}{code}
This ensures the loop terminates before trying to do the next token. There is probably a more elegant approach but this is the least intrusive.

 

> camel-aws2-sqs should use pagination for deciding which aws sqs queues it should create
> ---------------------------------------------------------------------------------------
>
>                 Key: CAMEL-16287
>                 URL: https://issues.apache.org/jira/browse/CAMEL-16287
>             Project: Camel
>          Issue Type: Bug
>    Affects Versions: 2.25.3, 3.8.0
>            Reporter: Gyorgy Abraham
>            Assignee: Andrea Cosentino
>            Priority: Major
>             Fix For: 3.11.7, 3.14.3, 3.17.0
>
>
> There is an auto create queue feature for aws-sqs in camel-sqs component. Upon camel context creation / route building, it issues an API call to list all sqs queues, so it skips the creation for already existing ones. If camel context has a for example consumer route for an sqs queue that is not listed in the response of this API call, camel will try to to create it.
> However, this API only lists the first 1000 queues in the current account. If there are more then 1000 queues, camel-sqs will try to create a new one, and might fail. In our company's account, we have 1442 queues in our account and Camel tried to create a .fifo queue because it thought it wasnt already existing, resulting in application startup error.
> Relevant code: [https://github.com/apache/camel/blob/master/components/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Endpoint.java#L174]
> Relevant API docs at AWS: [https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/AmazonSQS.html#listQueues-com.amazonaws.services.sqs.model.ListQueuesRequest-]
> Solution is straightforward: use pagination and load all queues.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)