[jira] [Updated] (CAMEL-23952) camel-langchain4j-agent: support executeToolsConcurrently by making the Camel tool executor exchange-safe

["Federico Mariani (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-23952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Federico Mariani updated CAMEL-23952:
-------------------------------------
    Description: 
LangChain4j's {{AiServices.executeToolsConcurrently()}} / {{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round trip in parallel. For agents wired to several independent tools (MCP servers, Camel route tools, custom tools) this cuts round-trip latency from the *sum* of the tool durations to the *max* — e.g. an LLM requesting weather + user profile + inventory in one turn.

The option cannot be exposed today because the Camel {{ToolExecutor}} in {{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for concurrent execution: it mutates the *shared live exchange* (writes tool-argument headers, {{CamelToolName}}, and the body on the exchange being processed, then runs the tool route on it). Two tools executing concurrently would corrupt each other's state.

*Proposal* (two steps, second depends on first):
# Make the Camel tool executor exchange-safe: run each tool invocation on its own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern camel-langchain4j-tools already uses), never on the live exchange. This overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel route tools" — the exception-propagation bug has the same root cause. Define what flows back to the main exchange: with sequential execution the current header side-effects can be preserved via {{copyResults}}; with concurrent execution header merge order is ambiguous, so document that tool-route header side-effects are not propagated (or last-wins) in concurrent mode.
# Expose the option on {{AgentConfiguration}}: {{withExecuteToolsConcurrently()}} and {{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in {{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given, prefer a thread pool from the CamelContext's {{ExecutorServiceManager}} (managed thread pool profile, proper lifecycle and naming) over LangChain4j's internal default.

Related: CAMEL-23944 (prerequisite), CAMEL-23928.

_This issue was drafted by Claude Code on behalf of Federico Mariani._

  was:
LangChain4j's {{AiServices.executeToolsConcurrently()}} / {{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round trip in parallel. For agents wired to several independent tools (MCP servers, Camel route tools, custom tools) this cuts round-trip latency from the *sum* of the tool durations to the *max* — e.g. an LLM requesting weather + user profile + inventory in one turn.

The option cannot be exposed today because the Camel {{ToolExecutor}} in {{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for concurrent execution: it mutates the *shared live exchange* (writes tool-argument headers, {{CamelToolName}}, and the body on the exchange being processed, then runs the tool route on it). Two tools executing concurrently would corrupt each other's state.

*Proposal* (two steps, second depends on first):
# Make the Camel tool executor exchange-safe: run each tool invocation on its own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern camel-langchain4j-tools already uses), never on the live exchange. This overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel route tools" — the exception-propagation bug has the same root cause. Define what flows back to the main exchange: with sequential execution the current header side-effects can be preserved via {{copyResults}}; with concurrent execution header merge order is ambiguous, so document that tool-route header side-effects are not propagated (or last-wins) in concurrent mode.
# Expose the option on {{AgentConfiguration}}: {{withExecuteToolsConcurrently()}} and {{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in {{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given, prefer a thread pool from the CamelContext's {{ExecutorServiceManager}} (managed thread pool profile, proper lifecycle and naming) over LangChain4j's internal default.

Related: "camel-langchain4j-agent: ToolExecutionErrorHandler and compensateOnToolErrors never invoked for Camel route tools" (prerequisite), CAMEL-23928.

_This issue was drafted by Claude Code on behalf of Federico Mariani._


> camel-langchain4j-agent: support executeToolsConcurrently by making the Camel tool executor exchange-safe
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-23952
>                 URL: https://issues.apache.org/jira/browse/CAMEL-23952
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-langchain4j-agent
>            Reporter: Federico Mariani
>            Priority: Major
>
> LangChain4j's {{AiServices.executeToolsConcurrently()}} / {{executeToolsConcurrently(Executor)}} runs all tool calls of one LLM round trip in parallel. For agents wired to several independent tools (MCP servers, Camel route tools, custom tools) this cuts round-trip latency from the *sum* of the tool durations to the *max* — e.g. an LLM requesting weather + user profile + inventory in one turn.
> The option cannot be exposed today because the Camel {{ToolExecutor}} in {{LangChain4jAgentProducer.createCamelToolProvider}} is not safe for concurrent execution: it mutates the *shared live exchange* (writes tool-argument headers, {{CamelToolName}}, and the body on the exchange being processed, then runs the tool route on it). Two tools executing concurrently would corrupt each other's state.
> *Proposal* (two steps, second depends on first):
> # Make the Camel tool executor exchange-safe: run each tool invocation on its own {{ExchangeHelper.createCopy}} of a baseline exchange (the pattern camel-langchain4j-tools already uses), never on the live exchange. This overlaps with the fix for "ToolExecutionErrorHandler never invoked for Camel route tools" — the exception-propagation bug has the same root cause. Define what flows back to the main exchange: with sequential execution the current header side-effects can be preserved via {{copyResults}}; with concurrent execution header merge order is ambiguous, so document that tool-route header side-effects are not propagated (or last-wins) in concurrent mode.
> # Expose the option on {{AgentConfiguration}}: {{withExecuteToolsConcurrently()}} and {{withExecuteToolsConcurrently(Executor)}}, wired null-guarded in {{AbstractAgent.configureBuilder()}}. When no explicit {{Executor}} is given, prefer a thread pool from the CamelContext's {{ExecutorServiceManager}} (managed thread pool profile, proper lifecycle and naming) over LangChain4j's internal default.
> Related: CAMEL-23944 (prerequisite), CAMEL-23928.
> _This issue was drafted by Claude Code on behalf of Federico Mariani._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)