[jira] [Assigned] (CAMEL-23602) threads EIP does not respect maxQueueSize when using virtual threads

["Federico Mariani (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-23602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Federico Mariani reassigned CAMEL-23602:
----------------------------------------

    Assignee: Federico Mariani

> threads EIP does not respect maxQueueSize when using virtual threads
> --------------------------------------------------------------------
>
>                 Key: CAMEL-23602
>                 URL: https://issues.apache.org/jira/browse/CAMEL-23602
>             Project: Camel
>          Issue Type: Bug
>          Components: came-core
>    Affects Versions: 4.20.0
>            Reporter: Simon Rasmussen
>            Assignee: Federico Mariani
>            Priority: Major
>
> We are looking into switching from platform threads to virtual threads. We have multiple routes, consuming from SQS queues. To achieve high throughput today, we use threads EIP like this:
>  
> {code:java}
> from(someSqsQueue?&waitTimeSeconds=10&maxMessagesPerPoll=10&extendMessageVisibility=true).threads().maxQueueSize(50).bean(service);{code}
> as you can see, we use the threads EIP to do the actual processing. This way, we have a single consumer thread, using long poll, to fetch up to 10 messages at a time, and offload them to a pool of threads to get processed. The max queue size is set relatively low, because if it gets full, then extendMessageVisibility kicks in, and we don’t want to do this across a too large set of messages. It also has the benefit that the polling thread will pick up processing, effectively backing off from pulling in more load.
> Now, if we switch to using virtual threads, we observe that the max queue size is ignored, and we keep pulling in messages unlimited – isn’t this a bug in threads EIP when combined with virtual threads?
> I can use a custom executor service with threads to overcome this, but is that really the best way?
> Our motivation for switching to virtual threads is due to having too many platform threads when the number of consumers using this pattern increases within an application.
> see also: https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/Unbounded.20threads.20EIP.20when.20using.20virtual.20threads.3F/with/596960264



--
This message was sent by Atlassian Jira
(v8.20.10#820010)