[jira] [Commented] (CAMEL-16829) camel-core - Stuck processing with nested parallel splits and custom thread pool

["Frank Müller (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-16829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17632385#comment-17632385 ] 

Frank Müller commented on CAMEL-16829:
--------------------------------------

I just ran into the same problem, although my snippet to reliably reproduce the Problem isn't quite as brief: [https://github.com/TxKroar/camel-repro |https://github.com/TxKroar/camel-repro] (I took the liberty to post it here in addition to what Samuel provided on the off chance that it is somehow useful while fixing this issue)

I compared it with the snippet provided with Samuel and it is indeed the exact same problem:

Just as with the example provided by Samuel, the bug happens when the ThreadPool used by camel is exhausted and the Caller Thread is used for further processing: the {{Worker }}in the {{DefaultReactiveExecutor }}of the Caller Thread has {{running }}set to {{true }}and then {{DefaultReactiveExecutor#scheduleMain }}gets called further down the pipeline - the provided {{Runnable }}never gets called and thus the Caller Thread gets stuck in {{DefaultAsyncProcessorAwaitManager#await.}}

 

> camel-core - Stuck processing with nested parallel splits and custom thread pool
> --------------------------------------------------------------------------------
>
>                 Key: CAMEL-16829
>                 URL: https://issues.apache.org/jira/browse/CAMEL-16829
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 3.11.0
>            Reporter: Samuel Padou
>            Assignee: Claus Ibsen
>            Priority: Major
>             Fix For: 3.x
>
>
> The cause of this issue is difficult to pinpoint, but running the following code, the route will be stuck indefinitely in processing:
> {code:java}
> @Component
> public static class Route extends RouteBuilder {
> 	@Override
> 	public void configure() throws Exception {
> 		final var executorService = new ThreadPoolBuilder(getContext())
> 				.poolSize(1)
> 				.maxPoolSize(1)
> 				.maxQueueSize(0)
> 				.build("inner");
> 		from("direct:main")
> 				.split(body()).parallelProcessing()
> 				.to("direct:inner?synchronous=true");
> 		from("direct:inner")
> 				.split(body()).executorService(executorService)
> 				.log("${body}");
> 	}
> }
> @Component
> public static class Runner implements CommandLineRunner {
> 	@Autowired
> 	private CamelContext camelContext;
> 	@Autowired
> 	private ProducerTemplate producerTemplate;
> 	@Override
> 	public void run(String... args) {
> 		final var exchange = new DefaultExchange(camelContext);
> 		exchange.getIn().setBody(List.of(List.of("0-0", "0-1"), List.of("1-0", "1-1")));
> 		producerTemplate.send("direct:main", exchange);
> 	}
> }
> {code}
> There are two important parts to reproduce the issue:
> * The forward to direct:inner with synchronous=true. This will cause an await here and it is the point where the route processing is stuck.
> * The inner executor, with a small pool and no queue, which will trigger the default CallerRuns rejection policy and run the split in the original thread instead of as new one.
> The cause of the stuck await seems to be linked to the way the AsyncProcessorAwaitManager and DefaultReactiveExecutor.executeFromQueue() interacts. Here the await callback to decrement the latch have been pushed in a back queue of the reactive worker (probably by a scheduleMain in the middle), but the executeFromQueue does not process the worker back queues so the callback is not executed and we are stuck on the await.
> I'm not sure what the solution is here, maybe the executeFromQueue should process back queues, or the CallerRuns rejection policy is just not supported in the first place but it should probably not be the default then.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)