[jira] [Commented] (CAMEL-19060) Scalability ceiling

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-19060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720295#comment-17720295 ] 

Claus Ibsen commented on CAMEL-19060:
-------------------------------------

TODO: The LongAdder

I think this may break stuff, I can now see jbang have incorrect inflight exchanges

out in 5 seconds. Inflights per route: [route2 = 26]
2023-05-07 13:53:14.719 INFO 39678 --- [ - ShutdownTask] impl.engine.DefaultShutdownStrategy : Waiting as there are still 26 inflight and pending exchanges to complete, timeout in 4 seconds. Inflights per route: [route2 = 26]
2023-05-07 13:53:15.721 INFO 39678 --- [ - ShutdownTask] impl.engine.DefaultShutdownStrategy : Waiting as there are still 26 inflight and pending exchanges to complete, timeout in 3 seconds. Inflights per route: [route2 = 26]
2023-05-07 13:53:16.727 INFO 39678 --- [ - ShutdownTask] impl.engine.DefaultShutdownStrategy : Waiting as there are still 26 inflight and pending exchanges to complete, timeout in 2 seconds. Inflights per route: [route2 = 26]
2023-05-07 13:53:17.731 INFO 39678 --- [ - ShutdownTask] impl.engine.DefaultShutdownStrategy : Waiting as there are still 26 inflight and pending exchanges to complete, timeout in 1 seconds. Inflights per route: [route2 = 26]

> Scalability ceiling
> -------------------
>
>                 Key: CAMEL-19060
>                 URL: https://issues.apache.org/jira/browse/CAMEL-19060
>             Project: Camel
>          Issue Type: Task
>    Affects Versions: 3.18.5, 4.0-M1
>            Reporter: Otavio Rodolfo Piske
>            Assignee: Otavio Rodolfo Piske
>            Priority: Major
>         Attachments: perf-stats.txt, stats.txt
>
>
> When using the Seda producer/consumer on a large core count machine, it seems that Camel's throughput peak at around 9 producer / 9 consumer threads and after that starts to decrease [1]. This seems to indicate some kind of coherence issue - the cost to keep the threads coherent exceeds the system's ability to handle the load / use the system's free capacity.
>  
> 1. This, of course, taking in consideration that the thread pool had been adjusted and the machine has enough capacity. Both of which were true.
> The attachments contains `perf` reports of the analysis of Camel 3.18.5, but similar patterns also happen w/ 4.0.0-M1. In particular, note the low "Instruction per cycle" reported on the attachment [^perf-stats.txt] and the high incidence of Last Level Cache (LLC) misses to Remote Cache (HITM) in the attachment [^stats.txt].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)