[jira] [Updated] (CAMEL-24977) Add provider-agnostic semantic evaluation across Camel EIPs and error handling

["Luigi De Masi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-24977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luigi De Masi updated CAMEL-24977:
----------------------------------
    Description: 
h2. Motivation

Provide a provider-agnostic semantic evaluation API and Camel language so route authors can ask named questions about message content and use the resulting boolean decisions, categories, or scores across EIPs. For example, classify a support message, assess urgency, decide whether another attempt is worthwhile after a failure, validate whether an output addresses a request, or check whether a permitted action serves an approved task. The reusable capability is semantic evaluation; routing, validation, error handling and contextual action checks are applications of that capability.

CAMEL-24871 added the Jev integration. This proposal builds on that work by separating the common question/evaluation contract from provider-specific implementations and using existing Camel EIP expression and predicate APIs. Jev can implement the common SPI, and other hosted or local classifiers can implement it without requiring Jev wire compatibility.

The desired authoring experience is declarative: named questions in the same YAML file as routes, automatic provider-adapter discovery, and direct use through Camel's language integration. Route authors should not need to instantiate a predicate or adapter bean for the normal case.

This is a design proposal. The syntax below illustrates the intended behavior; public names, packaging and exact DSL/schema details require community review. It does not claim that the proposed semantic language is already available.

Keep the artifact name {{camel-semantic}} and language identifier {{semantic}}. Use *Semantic Evaluation* as the public catalog title and describe it as: "Evaluate named questions about message content to produce boolean decisions, categories and scores through provider adapters."

h2. Common semantic evaluation contract

* Define provider-independent question and result types for categorical Choice, yes/no judgments (including probability-to-boolean decision policy), and scoring against descriptive ordered levels. Preserve the useful shape of Jev Choice/Noul/Score through its adapter; final common type names should not force other providers to adopt Jev terminology.
* Separate the selected category/value, probabilities, provider confidence, uncertainty policy and operational errors. A probability, a confidence measure and a guarantee of correctness are not interchangeable. Preserve provider metadata where useful without requiring all providers to report the same optional fields.
* A provider component advertises an adapter implementing the common SPI. The adapter maps questions/state to its backend, validates supported capabilities, and maps typed results back to the common contract.
* Expose evaluations through Camel's existing {{Language}}, {{Expression}} and {{Predicate}} contracts. Predicate use requires a boolean decision or explicit comparison/threshold policy; a category string is not implicitly a predicate.
* Keep provider credentials, model selection, transport/inference configuration, and provider-specific resources with the implementing component. Define thread safety, bounded evaluation, cancellation and lifecycle behavior. Synchronous Predicate/Expression evaluation must document any blocking inference call.

h2. Named questions and state in YAML

Support a dedicated top-level {{semantic.question}} declaration from the initial version, alongside route definitions. The following is proposed syntax:

{code}
- semantic:
    question:
      department:
        type: choice
        state: ${header.myState}
        instructions: Which department should handle this message?
        criteria:
          billing: Invoices, payments, and refunds
          technical: Bugs, outages, and technical problems
          other: Everything else

- route:
    id: ticket-routing
    from:
      uri: direct:tickets
      steps:
        - setProperty:
            name: department
            expression:
              language:
                language: semantic
                expression: "ref:department"
        - choice:
            when:
              - expression:
                  simple:
                    expression: "${exchangeProperty.department} == 'billing'"
                steps:
                  - to: direct:billing
              - expression:
                  simple:
                    expression: "${exchangeProperty.department} == 'technical'"
                steps:
                  - to: direct:technical
            otherwise:
              steps:
                - to: direct:general
{code}

Each question owns its input-state selector. An optional language default can be configured:

{code}
camel.language.semantic.default-state=${body}
{code}

Proposed precedence: question {{state}}, then configured {{default-state}}, then built-in {{${body}}}. Use Camel Simple expressions for these selectors initially. Compile/initialize selectors during route setup and evaluate against the current exchange at runtime. Only an absent selector inherits the default: a malformed selector or an explicitly selected header that evaluates to null must not silently cause different data to be classified. Input values are data, not recursively evaluated expressions.

Question declarations need loader, validation, schema and tooling support. Provide the YAML resolver from {{camel-semantic}} through the YAML DSL extension mechanism; the core YAML DSL must not depend on the semantic component. Keep YAML support optional for Java applications using semantic evaluation. Resolve references before traffic where possible; reject duplicate names, unknown references and unsupported question capabilities. Define behavior across multiple route resources and reload without retaining obsolete definitions. Loading/validating declarations must not perform inference.

h2. Adapter discovery and explicit selection

The language must not require a provider endpoint URI such as {{jev:decisions}}. With exactly one advertised adapter implementation available, discover and select it automatically. With none, report a missing provider; with multiple, report the candidates and require explicit selection. Do not choose by classpath order or construct every provider during discovery.

Allow an explicit reference to an existing registered instance:

{code}
camel.language.semantic.adapter=mySemanticAdapter
{code}

Alternatively, accept a fully qualified class name:

{code}
camel.language.semantic.adapter=com.example.MySemanticAdapter
{code}

The example class is illustrative. Use plain bean or class names, without {{#bean:}} or {{#class:}} prefixes. Registry lookup takes precedence over class resolution; this uses normal property binding without extending the core property-configurer SPI. For a class-selected or discovered adapter, the semantic infrastructure should resolve and type-check the class through Camel, instantiate it using the context Injector, register it once for the language configuration, and integrate its lifecycle. Reuse referenced beans without duplicating their existing lifecycle ownership. Define registry ownership, collision handling and startup-failure cleanup; binding an object does not itself establish service lifecycle management.

Provider configuration must remain effective even when the route contains no provider endpoint. For example, a Jev adapter must honor the configured Jev component's credentials and connection options. A manually declared adapter bean is an optional customization path, not a prerequisite for normal operation.

h2. Choice integration through the existing model

Keep Camel's existing Choice, When and expression models unchanged. Boolean questions can be used directly as ordinary when predicates. For category questions, evaluate once using Set Property or Set Header and compare the stored category using ordinary Choice predicates:

{code:java}
from("direct:tickets")
    .setProperty("department").language("semantic", "ref:department")
    .choice()
        .when(exchangeProperty("department").isEqualTo("billing"))
            .to("direct:billing")
        .when(exchangeProperty("department").isEqualTo("technical"))
            .to("direct:technical")
        .otherwise()
            .to("direct:general")
    .end();
{code}

* Perform one evaluation each time execution reaches the Set Property or Set Header step. Branch predicates reuse the stored result without invoking the provider again.
* Place that evaluation step inside a loop or retry path when changed state must be reevaluated. Nested decisions can use distinct properties to retain independent results. There is no implicit exchange-wide semantic-result cache.
* Valid categories without an explicit matching branch follow ordinary Choice otherwise behavior. Inference and invalid-response failures follow Camel error handling; uncertainty requires an explicit policy.
* Keep Choice precondition behavior unchanged. Message-dependent semantic predicates belong in runtime routes, not in startup preconditions evaluated against a dummy exchange.
* Use existing Java, XML and YAML expression/predicate forms. No Choice selector, literal when-value syntax, core EIP model changes or new routing EIP is included in this issue.

h2. Evaluation across other EIPs

The following are candidate uses, not a requirement to introduce semantic-specific changes to every EIP.

|| EIP / integration point || Example question or benefit || Integration ||
| Filter | Is this message relevant to this workflow? | A boolean predicate. Existing regression tests already cover semantic filtering. |
| Validate | Does this bug report describe an actionable problem? | A boolean decision supplies the validation predicate; negative decisions use normal validation failure handling. |
| Set Header / Set Property | Attach department, sentiment or urgency to a message. | Store a category or score for explicit reuse by subsequent processors. |
| Aggregate: grouping | Group feedback by delivery, pricing or usability. | Use a category as part of the correlation key, retaining tenant/case boundaries and bounded aggregation windows where needed. |
| Aggregate: completion | Does the accumulated conversation contain enough information to proceed? | Evaluate a boolean completion predicate against selected aggregate state, alongside a size or timeout completion condition. |
| Recipient List | Send a complaint to support and the relevant product team. | Map a category, or multiple independent decisions, to configured recipients. |
| Routing Slip | Select a document-processing sequence. | Map a document category to a predefined sequence of endpoints. |
| Enrich | Choose the appropriate knowledge source for a support request. | Map a topic to a configured enrichment endpoint; ordinary code retrieves and merges the additional data. |
| Loop | Continue refinement while the updated output remains incomplete. | Reevaluate a semantic predicate against current state, with an explicit iteration/time budget. |
| Sort | Order a batch of tickets by assessed urgency. | Score each item once, then compare stored scores with a deterministic comparator. |
| On Exception / retryWhile | Given this failure, is another attempt worthwhile? | Use a boolean predicate with an explicit retry budget checked before inference. {{retryWhile}} replaces the normal {{maximumRedeliveries}} decision; the predicate must enforce the cap itself. A negative decision or exhausted budget follows the configured escalation path. |
| Contextual action validation | Does this proposed action serve the approved task? | Apply existing identity, permission and tenant checks first, then Validate with a semantic predicate before the action. Negative, uncertain or failed evaluations must not execute the action. No dedicated semantic {{AuthorizationPolicy}} implementation is required. |
| Intercept / Intercept Send To Endpoint | Does this outgoing message require review? | Use {{onWhen}} to conditionally intercept processing or divert a send across selected routes. When enforcing a gate, configure interception to prevent the original send on rejection. |
| Dynamic Router | Given the updated state, which processing step should run next? | Map a category to a configured endpoint after each step. Enforce a hop budget and explicitly return {{null}} from the routing expression to finish. |
| Split with Filter or Validate | Which individual records are relevant or acceptable? | Split an existing collection, then evaluate each item. Semantic evaluation does not itself extract or generate the collection. |
| Poll Enrich / To Dynamic | Which configured source or service should handle this content? | Classify first, then map the label to an approved endpoint. This extends the destination-mapping pattern used for Enrich. |
| Throttle | Is this request routine or expensive to process? | Classify into configured workload groups, then apply deterministic rate or concurrency limits. The model does not invent limits. |
| On Completion | Does this completed interaction warrant follow-up? | Evaluate for auditing, review or follow-up processing. This cannot prevent actions already performed. |

Validate, metadata assignment and Aggregate can consume Predicate/Expression results directly. Recipient List, Routing Slip and Enrich require a mapping from semantic labels to route-author-defined destinations. The classifier does not invent endpoint addresses or processing steps.

For aggregate completion, the aggregation strategy must expose the relevant accumulated content; the classifier does not implicitly remember earlier exchanges. The default completion predicate sees the aggregated exchange, while {{eagerCheckCompletion}} uses the incoming exchange. State selection must match that behavior.

For Sort, the EIP expression selects the collection and the comparator determines ordering. Inference belongs before comparison. For loops or changed input, explicit result reuse must not hide relevant state changes.

Bounded retry and contextual action validation use existing Camel hooks and are part of the focused documentation and integration-test scope. Prepare current failure context in {{onExceptionOccurred}}, before the retry predicate; {{onRedelivery}} runs later. A retry-predicate evaluation error propagates as an error and does not automatically execute the normal escalation route. For contextual action validation, keep authentication, permissions and tenant boundaries authoritative; semantic evaluation is an additional check, and its failures must prevent execution.

h2. Broader Camel applications

* *AI tool and MCP routes:* check whether a proposed tool action fits the approved task before executing it, alongside existing permissions. Camel exposes tools through ordinary routes, so contextual validation can be applied there. See [Camel's tool authorization example|https://camel.apache.org/blog/2026/09/securing-ai-agent-tools/].
* *Input and output quality checks:* ask "Is this request sufficiently specified?" or "Does this answer address the request?" Route-level validation can surround an AI component. Direct integration into LangChain4j's internal guardrail interfaces would require an additional bridge; the semantic language does not provide that bridge. See [LangChain4j Agent guardrail support|https://camel.apache.org/components/4.22.x/langchain4j-agent-component.html].
* *Retrieval relevance and ranking:* evaluate retrieved documents against the question, filter irrelevant passages, and score candidates before deterministic sorting. This combines Split, Filter, Set Property and Sort; it complements retrieval rather than implementing a search engine.

The additional combinations are integration candidates inferred from existing extension points, not a claim that all combinations have been tested. Evaluation quality depends on the provider and question. Keep candidate exploration distinct from the initial regression coverage below; no semantic-specific changes to the core EIP models are required.

h2. Acceptance criteria and initial coverage

# A common provider-independent question/result contract and adapter SPI support direct semantic Expression/Predicate use without requiring manual predicate instances.
# A YAML file can declare named questions under {{semantic.question}} and reference them from routes. Loader, schema, validation and tooling support agree on the proposed configuration.
# Adapter discovery, bean-reference override and class-name override are tested, including missing/ambiguous providers, wrong types, instance reuse, configuration and lifecycle behavior.
# Question state overrides the language default; omitting both uses the body. Invalid selectors and missing selected state produce useful errors without silent input substitution.
# Choice integration uses the unchanged EIP model: store a category once per evaluation step and route with ordinary predicates. Tests cover invocation counts, first/later/otherwise branches, preservation of message content and reevaluation after state changes in a loop. Boolean questions remain usable directly as predicates.
# Demonstrate Choice, Filter, Validate, Set Header / Set Property, Aggregate, Recipient List, Routing Slip, Enrich, bounded Loop and Sort integration. Document the additional candidate integrations and their mapping/state requirements without assuming every EIP needs a core change.
# Document and test bounded semantic retry: approval followed by success, negative decision and budget exhaustion leading to escalation, refreshed failure state, no inference after the budget is exhausted, and propagation of timeout, malformed-response and uncertainty errors without another attempt.
# Document and test contextual action validation: ordinary permission denial prevents inference and action; a permitted action proceeds only after a positive semantic decision; negative, uncertain and failed evaluations prevent execution. Use existing Validate and authorization services without introducing a dedicated semantic authorization policy.
# Provide a Jev adapter and validate the contract with deterministic test adapters, including a second implementation and capability mismatch cases. Do not claim interchangeability of model quality, probabilities or calibration.
# Tests cover meaningful result mappings, invocation counts, threshold/uncertainty policy, explicit result reuse, changed-state reevaluation, timeouts/failures and preservation of message content. Routine tests require no live model credentials.
# Document the supported property namespaces and runtime configuration paths. Coordinate with the generic-language starter work in CAMEL-24913 and CAMEL-24914 rather than assuming language configuration is generated automatically.

h2. Boundaries and references

Choosing a local inference runtime (pure JVM/WASM versus native inference), model weights, or a particular hosted service is outside this proposal's initial decision. A provider can implement Jev API compatibility, but the common SPI does not require it and does not imply equivalent proprietary model behavior or calibration. The SPI's final package, discovery metadata and public syntax remain open to design review.

Related work:
* CAMEL-24871 — Jev component and semantic EIP integration background.
* CAMEL-24913 — Jev Spring Boot starter configuration.
* CAMEL-24914 — Starter generation for languages backed by the generic model.
* CAMEL-17755 — Existing Choice precondition design.
* [Camel Language SPI|https://github.com/apache/camel/blob/main/core/camel-api/src/main/java/org/apache/camel/spi/Language.java]
* [Choice model|https://github.com/apache/camel/blob/main/core/camel-core-model/src/main/java/org/apache/camel/model/ChoiceDefinition.java]
* [Aggregate EIP|https://camel.apache.org/components/next/eips/aggregate-eip.html]
* [Intercept|https://camel.apache.org/components/next/eips/intercept.html]
* [Dynamic Router EIP|https://camel.apache.org/components/next/eips/dynamicRouter-eip.html]
* [On Completion|https://camel.apache.org/manual/oncompletion.html]
* [Retry predicates|https://camel.apache.org/manual/exception-advanced.html]
* [System One Models|https://systemonemodels.org/]

_AI-generated by Codex on behalf of [luigidemasi|https://github.com/luigidemasi]._


  was:
h2. Motivation

Provide a provider-agnostic semantic evaluation API and Camel language so route authors can ask named questions about message content and use the resulting boolean decisions, categories, or scores across EIPs. For example, a support message can be classified as billing, technical, or other; a separate question can assess urgency or whether the message contains an actionable request.

CAMEL-24871 added the Jev integration. This proposal builds on that work by separating the common question/evaluation contract from provider-specific implementations and using existing Camel EIP expression and predicate APIs. Jev can implement the common SPI, and other hosted or local classifiers can implement it without requiring Jev wire compatibility.

The desired authoring experience is declarative: named questions in the same YAML file as routes, automatic provider-adapter discovery, and direct use through Camel's language integration. Route authors should not need to instantiate a predicate or adapter bean for the normal case.

This is a design proposal. The syntax below illustrates the intended behavior; public names, packaging and exact DSL/schema details require community review. It does not claim that the proposed semantic language is already available.

h2. Common semantic evaluation contract

* Define provider-independent question and result types for categorical Choice, yes/no judgments (including probability-to-boolean decision policy), and scoring against descriptive ordered levels. Preserve the useful shape of Jev Choice/Noul/Score through its adapter; final common type names should not force other providers to adopt Jev terminology.
* Separate the selected category/value, probabilities, provider confidence, uncertainty policy and operational errors. A probability, a confidence measure and a guarantee of correctness are not interchangeable. Preserve provider metadata where useful without requiring all providers to report the same optional fields.
* A provider component advertises an adapter implementing the common SPI. The adapter maps questions/state to its backend, validates supported capabilities, and maps typed results back to the common contract.
* Expose evaluations through Camel's existing {{Language}}, {{Expression}} and {{Predicate}} contracts. Predicate use requires a boolean decision or explicit comparison/threshold policy; a category string is not implicitly a predicate.
* Keep provider credentials, model selection, transport/inference configuration, and provider-specific resources with the implementing component. Define thread safety, bounded evaluation, cancellation and lifecycle behavior. Synchronous Predicate/Expression evaluation must document any blocking inference call.

h2. Named questions and state in YAML

Support a dedicated top-level {{semantic.question}} declaration from the initial version, alongside route definitions. The following is proposed syntax:

{code}
- semantic:
    question:
      department:
        type: choice
        state: ${header.myState}
        instructions: Which department should handle this message?
        criteria:
          billing: Invoices, payments, and refunds
          technical: Bugs, outages, and technical problems
          other: Everything else

- route:
    id: ticket-routing
    from:
      uri: direct:tickets
      steps:
        - setProperty:
            name: department
            expression:
              language:
                language: semantic
                expression: "ref:department"
        - choice:
            when:
              - expression:
                  simple:
                    expression: "${exchangeProperty.department} == 'billing'"
                steps:
                  - to: direct:billing
              - expression:
                  simple:
                    expression: "${exchangeProperty.department} == 'technical'"
                steps:
                  - to: direct:technical
            otherwise:
              steps:
                - to: direct:general
{code}

Each question owns its input-state selector. An optional language default can be configured:

{code}
camel.language.semantic.default-state=${body}
{code}

Proposed precedence: question {{state}}, then configured {{default-state}}, then built-in {{${body}}}. Use Camel Simple expressions for these selectors initially. Compile/initialize selectors during route setup and evaluate against the current exchange at runtime. Only an absent selector inherits the default: a malformed selector or an explicitly selected header that evaluates to null must not silently cause different data to be classified. Input values are data, not recursively evaluated expressions.

Question declarations need loader, validation, schema and tooling support. Provide the YAML resolver from {{camel-semantic}} through the YAML DSL extension mechanism; the core YAML DSL must not depend on the semantic component. Keep YAML support optional for Java applications using semantic evaluation. Resolve references before traffic where possible; reject duplicate names, unknown references and unsupported question capabilities. Define behavior across multiple route resources and reload without retaining obsolete definitions. Loading/validating declarations must not perform inference.

h2. Adapter discovery and explicit selection

The language must not require a provider endpoint URI such as {{jev:decisions}}. With exactly one advertised adapter implementation available, discover and select it automatically. With none, report a missing provider; with multiple, report the candidates and require explicit selection. Do not choose by classpath order or construct every provider during discovery.

Allow an explicit reference to an existing registered instance:

{code}
camel.language.semantic.adapter=mySemanticAdapter
{code}

Alternatively, accept a fully qualified class name:

{code}
camel.language.semantic.adapter=com.example.MySemanticAdapter
{code}

The example class is illustrative. Use plain bean or class names, without {{#bean:}} or {{#class:}} prefixes. Registry lookup takes precedence over class resolution; this uses normal property binding without extending the core property-configurer SPI. For a class-selected or discovered adapter, the semantic infrastructure should resolve and type-check the class through Camel, instantiate it using the context Injector, register it once for the language configuration, and integrate its lifecycle. Reuse referenced beans without duplicating their existing lifecycle ownership. Define registry ownership, collision handling and startup-failure cleanup; binding an object does not itself establish service lifecycle management.

Provider configuration must remain effective even when the route contains no provider endpoint. For example, a Jev adapter must honor the configured Jev component's credentials and connection options. A manually declared adapter bean is an optional customization path, not a prerequisite for normal operation.

h2. Choice integration through the existing model

Keep Camel's existing Choice, When and expression models unchanged. Boolean questions can be used directly as ordinary when predicates. For category questions, evaluate once using Set Property or Set Header and compare the stored category using ordinary Choice predicates:

{code:java}
from("direct:tickets")
    .setProperty("department").language("semantic", "ref:department")
    .choice()
        .when(exchangeProperty("department").isEqualTo("billing"))
            .to("direct:billing")
        .when(exchangeProperty("department").isEqualTo("technical"))
            .to("direct:technical")
        .otherwise()
            .to("direct:general")
    .end();
{code}

* Perform one evaluation each time execution reaches the Set Property or Set Header step. Branch predicates reuse the stored result without invoking the provider again.
* Place that evaluation step inside a loop or retry path when changed state must be reevaluated. Nested decisions can use distinct properties to retain independent results. There is no implicit exchange-wide semantic-result cache.
* Valid categories without an explicit matching branch follow ordinary Choice otherwise behavior. Inference and invalid-response failures follow Camel error handling; uncertainty requires an explicit policy.
* Keep Choice precondition behavior unchanged. Message-dependent semantic predicates belong in runtime routes, not in startup preconditions evaluated against a dummy exchange.
* Use existing Java, XML and YAML expression/predicate forms. No Choice selector, literal when-value syntax, core EIP model changes or new routing EIP is included in this issue.

h2. Evaluation across other EIPs

The following are candidate uses, not a requirement to introduce semantic-specific changes to every EIP.

|| EIP || Example benefit || Integration ||
| Validate | Does this bug report describe an actionable problem? | A boolean decision supplies the validation predicate; negative decisions use normal validation failure handling. |
| Set Header / Set Property | Attach department, sentiment or urgency to a message. | Store a category or score for explicit reuse by subsequent processors. |
| Aggregate: grouping | Group feedback by delivery, pricing or usability. | Use a category as part of the correlation key, retaining tenant/case boundaries and bounded aggregation windows where needed. |
| Aggregate: completion | Does the accumulated conversation contain enough information to proceed? | Evaluate a boolean completion predicate against selected aggregate state, alongside a size or timeout completion condition. |
| Recipient List | Send a complaint to support and the relevant product team. | Map a category, or multiple independent decisions, to configured recipients. |
| Routing Slip | Select a document-processing sequence. | Map a document category to a predefined sequence of endpoints. |
| Enrich | Choose the appropriate knowledge source for a support request. | Map a topic to a configured enrichment endpoint; ordinary code retrieves and merges the additional data. |
| Loop | Continue refinement while the updated output remains incomplete. | Reevaluate a semantic predicate against current state, with an explicit iteration/time budget. |
| Sort | Order a batch of tickets by assessed urgency. | Score each item once, then compare stored scores with a deterministic comparator. |

Validate, metadata assignment and Aggregate can consume Predicate/Expression results directly. Recipient List, Routing Slip and Enrich require a mapping from semantic labels to route-author-defined destinations. The classifier does not invent endpoint addresses or processing steps.

For aggregate completion, the aggregation strategy must expose the relevant accumulated content; the classifier does not implicitly remember earlier exchanges. The default completion predicate sees the aggregated exchange, while {{eagerCheckCompletion}} uses the incoming exchange. State selection must match that behavior.

For Sort, the EIP expression selects the collection and the comparator determines ordering. Inference belongs before comparison. For loops or changed input, explicit result reuse must not hide relevant state changes.

h2. Acceptance criteria and initial coverage

# A common provider-independent question/result contract and adapter SPI support direct semantic Expression/Predicate use without requiring manual predicate instances.
# A YAML file can declare named questions under {{semantic.question}} and reference them from routes. Loader, schema, validation and tooling support agree on the proposed configuration.
# Adapter discovery, bean-reference override and class-name override are tested, including missing/ambiguous providers, wrong types, instance reuse, configuration and lifecycle behavior.
# Question state overrides the language default; omitting both uses the body. Invalid selectors and missing selected state produce useful errors without silent input substitution.
# Choice integration uses the unchanged EIP model: store a category once per evaluation step and route with ordinary predicates. Tests cover invocation counts, first/later/otherwise branches, preservation of message content and reevaluation after state changes in a loop. Boolean questions remain usable directly as predicates.
# Demonstrate Choice, Filter, Validate, Set Property and Aggregate integration. Document the other candidate EIPs and their mapping/state requirements without assuming every EIP needs a core change.
# Provide a Jev adapter and validate the contract with deterministic test adapters, including a second implementation and capability mismatch cases. Do not claim interchangeability of model quality, probabilities or calibration.
# Tests cover meaningful result mappings, invocation counts, threshold/uncertainty policy, explicit result reuse, changed-state reevaluation, timeouts/failures and preservation of message content. Routine tests require no live model credentials.
# Document the supported property namespaces and runtime configuration paths. Coordinate with the generic-language starter work in CAMEL-24913 and CAMEL-24914 rather than assuming language configuration is generated automatically.

h2. Boundaries and references

Choosing a local inference runtime (pure JVM/WASM versus native inference), model weights, or a particular hosted service is outside this proposal's initial decision. A provider can implement Jev API compatibility, but the common SPI does not require it and does not imply equivalent proprietary model behavior or calibration. The SPI's final package, discovery metadata and public syntax remain open to design review.

Related work:
* CAMEL-24871 — Jev component and semantic EIP integration background.
* CAMEL-24913 — Jev Spring Boot starter configuration.
* CAMEL-24914 — Starter generation for languages backed by the generic model.
* CAMEL-17755 — Existing Choice precondition design.
* [Camel Language SPI|https://github.com/apache/camel/blob/main/core/camel-api/src/main/java/org/apache/camel/spi/Language.java]
* [Choice model|https://github.com/apache/camel/blob/main/core/camel-core-model/src/main/java/org/apache/camel/model/ChoiceDefinition.java]
* [Aggregate EIP|https://camel.apache.org/components/next/eips/aggregate-eip.html]
* [System One Models|https://systemonemodels.org/]

_AI-generated by Codex on behalf of [luigidemasi|https://github.com/luigidemasi]._


        Summary: Add provider-agnostic semantic evaluation across Camel EIPs and error handling  (was: Add provider-agnostic semantic evaluation across Camel EIPs)

> Add provider-agnostic semantic evaluation across Camel EIPs and error handling
> ------------------------------------------------------------------------------
>
>                 Key: CAMEL-24977
>                 URL: https://issues.apache.org/jira/browse/CAMEL-24977
>             Project: Camel
>          Issue Type: New Feature
>          Components: camel-ai, camel-core, camel-yaml-dsl, eip
>            Reporter: Luigi De Masi
>            Assignee: Luigi De Masi
>            Priority: Major
>
> h2. Motivation
> Provide a provider-agnostic semantic evaluation API and Camel language so route authors can ask named questions about message content and use the resulting boolean decisions, categories, or scores across EIPs. For example, classify a support message, assess urgency, decide whether another attempt is worthwhile after a failure, validate whether an output addresses a request, or check whether a permitted action serves an approved task. The reusable capability is semantic evaluation; routing, validation, error handling and contextual action checks are applications of that capability.
> CAMEL-24871 added the Jev integration. This proposal builds on that work by separating the common question/evaluation contract from provider-specific implementations and using existing Camel EIP expression and predicate APIs. Jev can implement the common SPI, and other hosted or local classifiers can implement it without requiring Jev wire compatibility.
> The desired authoring experience is declarative: named questions in the same YAML file as routes, automatic provider-adapter discovery, and direct use through Camel's language integration. Route authors should not need to instantiate a predicate or adapter bean for the normal case.
> This is a design proposal. The syntax below illustrates the intended behavior; public names, packaging and exact DSL/schema details require community review. It does not claim that the proposed semantic language is already available.
> Keep the artifact name {{camel-semantic}} and language identifier {{semantic}}. Use *Semantic Evaluation* as the public catalog title and describe it as: "Evaluate named questions about message content to produce boolean decisions, categories and scores through provider adapters."
> h2. Common semantic evaluation contract
> * Define provider-independent question and result types for categorical Choice, yes/no judgments (including probability-to-boolean decision policy), and scoring against descriptive ordered levels. Preserve the useful shape of Jev Choice/Noul/Score through its adapter; final common type names should not force other providers to adopt Jev terminology.
> * Separate the selected category/value, probabilities, provider confidence, uncertainty policy and operational errors. A probability, a confidence measure and a guarantee of correctness are not interchangeable. Preserve provider metadata where useful without requiring all providers to report the same optional fields.
> * A provider component advertises an adapter implementing the common SPI. The adapter maps questions/state to its backend, validates supported capabilities, and maps typed results back to the common contract.
> * Expose evaluations through Camel's existing {{Language}}, {{Expression}} and {{Predicate}} contracts. Predicate use requires a boolean decision or explicit comparison/threshold policy; a category string is not implicitly a predicate.
> * Keep provider credentials, model selection, transport/inference configuration, and provider-specific resources with the implementing component. Define thread safety, bounded evaluation, cancellation and lifecycle behavior. Synchronous Predicate/Expression evaluation must document any blocking inference call.
> h2. Named questions and state in YAML
> Support a dedicated top-level {{semantic.question}} declaration from the initial version, alongside route definitions. The following is proposed syntax:
> {code}
> - semantic:
>     question:
>       department:
>         type: choice
>         state: ${header.myState}
>         instructions: Which department should handle this message?
>         criteria:
>           billing: Invoices, payments, and refunds
>           technical: Bugs, outages, and technical problems
>           other: Everything else
> - route:
>     id: ticket-routing
>     from:
>       uri: direct:tickets
>       steps:
>         - setProperty:
>             name: department
>             expression:
>               language:
>                 language: semantic
>                 expression: "ref:department"
>         - choice:
>             when:
>               - expression:
>                   simple:
>                     expression: "${exchangeProperty.department} == 'billing'"
>                 steps:
>                   - to: direct:billing
>               - expression:
>                   simple:
>                     expression: "${exchangeProperty.department} == 'technical'"
>                 steps:
>                   - to: direct:technical
>             otherwise:
>               steps:
>                 - to: direct:general
> {code}
> Each question owns its input-state selector. An optional language default can be configured:
> {code}
> camel.language.semantic.default-state=${body}
> {code}
> Proposed precedence: question {{state}}, then configured {{default-state}}, then built-in {{${body}}}. Use Camel Simple expressions for these selectors initially. Compile/initialize selectors during route setup and evaluate against the current exchange at runtime. Only an absent selector inherits the default: a malformed selector or an explicitly selected header that evaluates to null must not silently cause different data to be classified. Input values are data, not recursively evaluated expressions.
> Question declarations need loader, validation, schema and tooling support. Provide the YAML resolver from {{camel-semantic}} through the YAML DSL extension mechanism; the core YAML DSL must not depend on the semantic component. Keep YAML support optional for Java applications using semantic evaluation. Resolve references before traffic where possible; reject duplicate names, unknown references and unsupported question capabilities. Define behavior across multiple route resources and reload without retaining obsolete definitions. Loading/validating declarations must not perform inference.
> h2. Adapter discovery and explicit selection
> The language must not require a provider endpoint URI such as {{jev:decisions}}. With exactly one advertised adapter implementation available, discover and select it automatically. With none, report a missing provider; with multiple, report the candidates and require explicit selection. Do not choose by classpath order or construct every provider during discovery.
> Allow an explicit reference to an existing registered instance:
> {code}
> camel.language.semantic.adapter=mySemanticAdapter
> {code}
> Alternatively, accept a fully qualified class name:
> {code}
> camel.language.semantic.adapter=com.example.MySemanticAdapter
> {code}
> The example class is illustrative. Use plain bean or class names, without {{#bean:}} or {{#class:}} prefixes. Registry lookup takes precedence over class resolution; this uses normal property binding without extending the core property-configurer SPI. For a class-selected or discovered adapter, the semantic infrastructure should resolve and type-check the class through Camel, instantiate it using the context Injector, register it once for the language configuration, and integrate its lifecycle. Reuse referenced beans without duplicating their existing lifecycle ownership. Define registry ownership, collision handling and startup-failure cleanup; binding an object does not itself establish service lifecycle management.
> Provider configuration must remain effective even when the route contains no provider endpoint. For example, a Jev adapter must honor the configured Jev component's credentials and connection options. A manually declared adapter bean is an optional customization path, not a prerequisite for normal operation.
> h2. Choice integration through the existing model
> Keep Camel's existing Choice, When and expression models unchanged. Boolean questions can be used directly as ordinary when predicates. For category questions, evaluate once using Set Property or Set Header and compare the stored category using ordinary Choice predicates:
> {code:java}
> from("direct:tickets")
>     .setProperty("department").language("semantic", "ref:department")
>     .choice()
>         .when(exchangeProperty("department").isEqualTo("billing"))
>             .to("direct:billing")
>         .when(exchangeProperty("department").isEqualTo("technical"))
>             .to("direct:technical")
>         .otherwise()
>             .to("direct:general")
>     .end();
> {code}
> * Perform one evaluation each time execution reaches the Set Property or Set Header step. Branch predicates reuse the stored result without invoking the provider again.
> * Place that evaluation step inside a loop or retry path when changed state must be reevaluated. Nested decisions can use distinct properties to retain independent results. There is no implicit exchange-wide semantic-result cache.
> * Valid categories without an explicit matching branch follow ordinary Choice otherwise behavior. Inference and invalid-response failures follow Camel error handling; uncertainty requires an explicit policy.
> * Keep Choice precondition behavior unchanged. Message-dependent semantic predicates belong in runtime routes, not in startup preconditions evaluated against a dummy exchange.
> * Use existing Java, XML and YAML expression/predicate forms. No Choice selector, literal when-value syntax, core EIP model changes or new routing EIP is included in this issue.
> h2. Evaluation across other EIPs
> The following are candidate uses, not a requirement to introduce semantic-specific changes to every EIP.
> || EIP / integration point || Example question or benefit || Integration ||
> | Filter | Is this message relevant to this workflow? | A boolean predicate. Existing regression tests already cover semantic filtering. |
> | Validate | Does this bug report describe an actionable problem? | A boolean decision supplies the validation predicate; negative decisions use normal validation failure handling. |
> | Set Header / Set Property | Attach department, sentiment or urgency to a message. | Store a category or score for explicit reuse by subsequent processors. |
> | Aggregate: grouping | Group feedback by delivery, pricing or usability. | Use a category as part of the correlation key, retaining tenant/case boundaries and bounded aggregation windows where needed. |
> | Aggregate: completion | Does the accumulated conversation contain enough information to proceed? | Evaluate a boolean completion predicate against selected aggregate state, alongside a size or timeout completion condition. |
> | Recipient List | Send a complaint to support and the relevant product team. | Map a category, or multiple independent decisions, to configured recipients. |
> | Routing Slip | Select a document-processing sequence. | Map a document category to a predefined sequence of endpoints. |
> | Enrich | Choose the appropriate knowledge source for a support request. | Map a topic to a configured enrichment endpoint; ordinary code retrieves and merges the additional data. |
> | Loop | Continue refinement while the updated output remains incomplete. | Reevaluate a semantic predicate against current state, with an explicit iteration/time budget. |
> | Sort | Order a batch of tickets by assessed urgency. | Score each item once, then compare stored scores with a deterministic comparator. |
> | On Exception / retryWhile | Given this failure, is another attempt worthwhile? | Use a boolean predicate with an explicit retry budget checked before inference. {{retryWhile}} replaces the normal {{maximumRedeliveries}} decision; the predicate must enforce the cap itself. A negative decision or exhausted budget follows the configured escalation path. |
> | Contextual action validation | Does this proposed action serve the approved task? | Apply existing identity, permission and tenant checks first, then Validate with a semantic predicate before the action. Negative, uncertain or failed evaluations must not execute the action. No dedicated semantic {{AuthorizationPolicy}} implementation is required. |
> | Intercept / Intercept Send To Endpoint | Does this outgoing message require review? | Use {{onWhen}} to conditionally intercept processing or divert a send across selected routes. When enforcing a gate, configure interception to prevent the original send on rejection. |
> | Dynamic Router | Given the updated state, which processing step should run next? | Map a category to a configured endpoint after each step. Enforce a hop budget and explicitly return {{null}} from the routing expression to finish. |
> | Split with Filter or Validate | Which individual records are relevant or acceptable? | Split an existing collection, then evaluate each item. Semantic evaluation does not itself extract or generate the collection. |
> | Poll Enrich / To Dynamic | Which configured source or service should handle this content? | Classify first, then map the label to an approved endpoint. This extends the destination-mapping pattern used for Enrich. |
> | Throttle | Is this request routine or expensive to process? | Classify into configured workload groups, then apply deterministic rate or concurrency limits. The model does not invent limits. |
> | On Completion | Does this completed interaction warrant follow-up? | Evaluate for auditing, review or follow-up processing. This cannot prevent actions already performed. |
> Validate, metadata assignment and Aggregate can consume Predicate/Expression results directly. Recipient List, Routing Slip and Enrich require a mapping from semantic labels to route-author-defined destinations. The classifier does not invent endpoint addresses or processing steps.
> For aggregate completion, the aggregation strategy must expose the relevant accumulated content; the classifier does not implicitly remember earlier exchanges. The default completion predicate sees the aggregated exchange, while {{eagerCheckCompletion}} uses the incoming exchange. State selection must match that behavior.
> For Sort, the EIP expression selects the collection and the comparator determines ordering. Inference belongs before comparison. For loops or changed input, explicit result reuse must not hide relevant state changes.
> Bounded retry and contextual action validation use existing Camel hooks and are part of the focused documentation and integration-test scope. Prepare current failure context in {{onExceptionOccurred}}, before the retry predicate; {{onRedelivery}} runs later. A retry-predicate evaluation error propagates as an error and does not automatically execute the normal escalation route. For contextual action validation, keep authentication, permissions and tenant boundaries authoritative; semantic evaluation is an additional check, and its failures must prevent execution.
> h2. Broader Camel applications
> * *AI tool and MCP routes:* check whether a proposed tool action fits the approved task before executing it, alongside existing permissions. Camel exposes tools through ordinary routes, so contextual validation can be applied there. See [Camel's tool authorization example|https://camel.apache.org/blog/2026/09/securing-ai-agent-tools/].
> * *Input and output quality checks:* ask "Is this request sufficiently specified?" or "Does this answer address the request?" Route-level validation can surround an AI component. Direct integration into LangChain4j's internal guardrail interfaces would require an additional bridge; the semantic language does not provide that bridge. See [LangChain4j Agent guardrail support|https://camel.apache.org/components/4.22.x/langchain4j-agent-component.html].
> * *Retrieval relevance and ranking:* evaluate retrieved documents against the question, filter irrelevant passages, and score candidates before deterministic sorting. This combines Split, Filter, Set Property and Sort; it complements retrieval rather than implementing a search engine.
> The additional combinations are integration candidates inferred from existing extension points, not a claim that all combinations have been tested. Evaluation quality depends on the provider and question. Keep candidate exploration distinct from the initial regression coverage below; no semantic-specific changes to the core EIP models are required.
> h2. Acceptance criteria and initial coverage
> # A common provider-independent question/result contract and adapter SPI support direct semantic Expression/Predicate use without requiring manual predicate instances.
> # A YAML file can declare named questions under {{semantic.question}} and reference them from routes. Loader, schema, validation and tooling support agree on the proposed configuration.
> # Adapter discovery, bean-reference override and class-name override are tested, including missing/ambiguous providers, wrong types, instance reuse, configuration and lifecycle behavior.
> # Question state overrides the language default; omitting both uses the body. Invalid selectors and missing selected state produce useful errors without silent input substitution.
> # Choice integration uses the unchanged EIP model: store a category once per evaluation step and route with ordinary predicates. Tests cover invocation counts, first/later/otherwise branches, preservation of message content and reevaluation after state changes in a loop. Boolean questions remain usable directly as predicates.
> # Demonstrate Choice, Filter, Validate, Set Header / Set Property, Aggregate, Recipient List, Routing Slip, Enrich, bounded Loop and Sort integration. Document the additional candidate integrations and their mapping/state requirements without assuming every EIP needs a core change.
> # Document and test bounded semantic retry: approval followed by success, negative decision and budget exhaustion leading to escalation, refreshed failure state, no inference after the budget is exhausted, and propagation of timeout, malformed-response and uncertainty errors without another attempt.
> # Document and test contextual action validation: ordinary permission denial prevents inference and action; a permitted action proceeds only after a positive semantic decision; negative, uncertain and failed evaluations prevent execution. Use existing Validate and authorization services without introducing a dedicated semantic authorization policy.
> # Provide a Jev adapter and validate the contract with deterministic test adapters, including a second implementation and capability mismatch cases. Do not claim interchangeability of model quality, probabilities or calibration.
> # Tests cover meaningful result mappings, invocation counts, threshold/uncertainty policy, explicit result reuse, changed-state reevaluation, timeouts/failures and preservation of message content. Routine tests require no live model credentials.
> # Document the supported property namespaces and runtime configuration paths. Coordinate with the generic-language starter work in CAMEL-24913 and CAMEL-24914 rather than assuming language configuration is generated automatically.
> h2. Boundaries and references
> Choosing a local inference runtime (pure JVM/WASM versus native inference), model weights, or a particular hosted service is outside this proposal's initial decision. A provider can implement Jev API compatibility, but the common SPI does not require it and does not imply equivalent proprietary model behavior or calibration. The SPI's final package, discovery metadata and public syntax remain open to design review.
> Related work:
> * CAMEL-24871 — Jev component and semantic EIP integration background.
> * CAMEL-24913 — Jev Spring Boot starter configuration.
> * CAMEL-24914 — Starter generation for languages backed by the generic model.
> * CAMEL-17755 — Existing Choice precondition design.
> * [Camel Language SPI|https://github.com/apache/camel/blob/main/core/camel-api/src/main/java/org/apache/camel/spi/Language.java]
> * [Choice model|https://github.com/apache/camel/blob/main/core/camel-core-model/src/main/java/org/apache/camel/model/ChoiceDefinition.java]
> * [Aggregate EIP|https://camel.apache.org/components/next/eips/aggregate-eip.html]
> * [Intercept|https://camel.apache.org/components/next/eips/intercept.html]
> * [Dynamic Router EIP|https://camel.apache.org/components/next/eips/dynamicRouter-eip.html]
> * [On Completion|https://camel.apache.org/manual/oncompletion.html]
> * [Retry predicates|https://camel.apache.org/manual/exception-advanced.html]
> * [System One Models|https://systemonemodels.org/]
> _AI-generated by Codex on behalf of [luigidemasi|https://github.com/luigidemasi]._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)