[jira] [Work started] (CAMEL-24687) camel-quickjs - every evaluation leaks a QuickJS module; 8 threads exhaust a 1 GB heap in 22 seconds

["Federico Mariani (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-24687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on CAMEL-24687 started by Federico Mariani.
------------------------------------------------
> camel-quickjs - every evaluation leaks a QuickJS module; 8 threads exhaust a 1 GB heap in 22 seconds
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-24687
>                 URL: https://issues.apache.org/jira/browse/CAMEL-24687
>             Project: Camel
>          Issue Type: Bug
>    Affects Versions: 4.23.0
>            Reporter: Federico Mariani
>            Assignee: Federico Mariani
>            Priority: Major
>
> QuickJS4J executes a module per {{invokeGuestFunction}} call and QuickJS keeps evaluated modules until its context is freed, so a camel-quickjs engine grows by about 12 KB of WebAssembly memory per evaluation and never shrinks.
> Reproduced twice with the 8-thread load test of the benchmark branch ({{GroovyLoadTest}}, variant {{quickjs}}, {{-Xms1g -Xmx1g}}): heap after young GC 496 MB at 5 s, 939 MB at 20 s, then {{java.lang.OutOfMemoryError}} in the producer threads at ~22 s, at about 4,000 msg/s. The GC log shows 919 MB live after a full GC, i.e. it is retained, not garbage.
> Proposed fix (implemented, tested, benchmarked): keep a handle on the engine's linear memory through the Endive memory factory and recycle a worker thread's engine (close it, create a fresh one on the next evaluation) once its memory exceeds {{engineMaxMemory}} (default 64 MB) or it has run {{engineMaxEvaluations}} (default 50,000) evaluations. With that, the same 5-minute run completes at 8,858 msg/s with no full GC and a stable heap. An upstream quickjs4j issue (a way to invoke a compiled function without re-evaluating a module) would remove the need for recycling.
> Related: CAMEL-23398 (component), apache/camel#25778.
> ----
> _Found during a benchmark review of camel-groovy and the other scripting languages (camel-performance-tests branch camel-groovy-perf, tests/camel-jmh/groovy-results, JMH 1.37, JDK 21, one M1 Pro laptop). A candidate fix with tests exists on the local apache/camel branch camel-perf-all-fixes; a PR follows._
> _Claude Code on behalf of Croway_



--
This message was sent by Atlassian Jira
(v8.20.10#820010)