[jira] [Commented] (CAMEL-17144) Split with aggregation strategy does not wait all substasks to be completed

["Thomas Sergent (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17437304#comment-17437304 ] 

Thomas Sergent commented on CAMEL-17144:
----------------------------------------

{quote}
 It's possible there's some bug here, but setting your scheduler to greedy=false is a workaround.
{quote}

Not at all in my case, I need to use greedy and because the behavior seems to be wrong I have created this issue. 
I suspect a bad behavior on split operation when reading the documentation all sub exchanged must be completed before the process to continue. It seems that nothing wait for the completion of all sub task. May be code issue or DSL issue ? 

{code}
I'm not sure how greedy=true makes sense for the scheduler component.
{code}

In my case, here is the use case. We need to process information based on a database in a way that if there is some data matching our query all must be procced asap. To avoid memory problem, the select on database uses a limit. So we use the scheduler component to fire an event every X minutes to call a direct route. This route enriches the exchange with database select, if there is no row selected, we can set the idle property to avoid the scheduler to fire at the end. In the other case, we can process all information and be sure that the scheduler will fire again the exchange to continue to process remaining information (if some exist). This greedy property is ideal in our use case ^^ because we are able to use the split with a thread profile in order to process data with multiple threads.


> Split with aggregation strategy does not wait all substasks to be completed
> ---------------------------------------------------------------------------
>
>                 Key: CAMEL-17144
>                 URL: https://issues.apache.org/jira/browse/CAMEL-17144
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-core
>    Affects Versions: 3.0.0, 3.12.0
>            Reporter: Thomas Sergent
>            Priority: Major
>         Attachments: parallel split test 2.txt, parallel split test.txt, test-camel-scheduler.zip
>
>
> Hi after trying to update from camel 2.25.4 to 3.12 I have a trouble with the usage of the split with parallelProcessing enabled. I cannot share my project but here is a little sample to reproduce the issue. 
>  
> {code:java}
> from("scheduler:testBug?initialDelay=1000&useFixedDelay=true&delay=5000&greedy=true")
> 		.to("direct:test");
> from("direct:test")
> 		.log("Before the processor the body must be empty : [${body}]")
> 		.process((exchange) -> {
> 			exchange.getIn().setBody(IntStream.range(0, 10).mapToObj(i -> "" + i).collect(Collectors.toList()));
> 		})
> 		.log("Exchange updated number form 0 to 9 : [${body}]")
> 		.split(body())
> 				.parallelProcessing(true)
> 				.log("The child exchange: ${body}")
> 		.end()
> 		.log("Split done: [${body}]")
> 		.setProperty(Exchange.SCHEDULER_POLLED_MESSAGES, simple("false"));
> {code}
>  
> With camel 2.25.X the expected behavior (to process each split before returning to the caller) is ok but not with the 3.X where the process loop indefinillty. 
>  
> Note: I have added a kill switch to avoid process to be hanged. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)