[jira] [Comment Edited] (CAMEL-24698) camel-jbang: follow-ups from the AI agent benchmarks (optional MCP args, CLI validate parity, eval placeholders, validator hints)

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-24698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114641#comment-18114641 ] 

Claus Ibsen edited comment on CAMEL-24698 at 9/13/26 9:55 AM:
--------------------------------------------------------------

Item 6 done on local branch fix/CAMEL-24698-bean-refs (commit ff62635d, stacked on the cli-validate branch, not pushed yet): SourceValidator.validateYamlBeanRefs checks ref, aggregationStrategy, bean, processor and the *Ref options against the beans: entries of the file, of the sibling YAML files, and the Java classes in the directory annotated with @BindToRegistry; #class:/#type:/#bean: values, placeholders and fully qualified class names are left alone. When a MyAggregator.java sits next to the route the message is:

{noformat}
Line 9: aggregationStrategy: bean 'myAggregator' is not declared: MyAggregator.java is in the directory, but a Java class is not a bean until it is declared; add
- beans:
  - name: myAggregator
    type: "#class:com.example.MyAggregator"
or reference the class directly: aggregationStrategy: "#class:com.example.MyAggregator"
{noformat}

camel_write_file and camel_validate_source pass their directory, camel validate yaml uses the file's directory; without a directory the check is not run, so a content-only validation is unchanged.


was (Author: davsclaus):
Item 6 done on local branch fix/CAMEL-24698-bean-refs (commit ff62635d, stacked on the cli-validate branch, not pushed yet): SourceValidator.validateYamlBeanRefs checks ref, aggregationStrategy, bean, processor and the *Ref options against the beans: entries of the file, of the sibling YAML files, and the Java classes in the directory annotated with @BindToRegistry; #class:/#type:/#bean: values, placeholders and fully qualified class names are left alone. When a MyAggregator.java sits next to the route the message is:

{noformat}
Line 9: aggregationStrategy: bean 'myAggregator' is not declared: MyAggregator.java is in the directory, but a Java class is not a bean until it is declared; add
- beans:
  - name: myAggregator
    type: "#class:com.example.MyAggregator"
or reference the class directly: aggregationStrategy: "#class:com.example.MyAggregator"
{noformat}

camel_write_file and camel_validate_source pass their directory, camel validate yaml uses the file's directory; without a directory the check is not run, so the TUI's content-only validation is unchanged.

> camel-jbang: follow-ups from the AI agent benchmarks (optional MCP args, CLI validate parity, eval placeholders, validator hints)
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-24698
>                 URL: https://issues.apache.org/jira/browse/CAMEL-24698
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-jbang, camel-yaml-dsl
>            Reporter: Claus Ibsen
>            Assignee: Claus Ibsen
>            Priority: Major
>
> Follow-ups from benchmarking AI agents (a frontier model and a 22 GB local model) editing Camel integrations through the Camel MCP tools, 2026-09-11/12. The bugs the benchmarks found first are tracked separately (CAMEL-24692, CAMEL-24693, CAMEL-24694, CAMEL-24695); this issue collects the remaining, smaller items so none of them is lost. Each can be fixed on its own.
> h3. 1. Older camel-jbang-mcp wrappers declare every argument required
> 69 of the 82 tools in camel-jbang-mcp mark every parameter as required in their {{@ToolArg}} declarations, so the Quarkus MCP server rejects any call that omits an optional one with "Missing required argument", and the documented defaults cannot be used. Example: {{camel_catalog_component_doc}} with only {{name}} fails with "Missing required argument: component". CAMEL-24695 fixed this for the eleven shared authoring tools and added a test that keeps their flags equal to the shared descriptors; the older wrappers ({{CatalogTools}}, {{DiagnoseTools}} except {{camel_error_diagnose}}, {{RuntimeTools}}, security, migration, OpenAPI, dependency tools) need {{required = false}} on every optional argument. A small model sends only the arguments it needs, so today it cannot use most of these tools.
> h3. 2. camel validate yaml does not check Simple expressions
> The CLI {{camel validate yaml}} validates the YAML DSL schema only. The Camel TUI's save-time validation and the shared {{camel_validate_source}} tool (CAMEL-24695, {{SourceValidator}} in camel-jbang-core) also check endpoint URIs, Simple expressions and {{camel.*}} properties. A frontier model shipped {{${padding(n)}}} (not a Simple function) past the CLI validator; the runtime caught it. {{camel validate yaml}} should use the shared {{SourceValidator}} so all three entry points report the same problems.
> h3. 3. Expression evaluation does not resolve property placeholders
> {{tui_eval_expression}} (now the shared {{camel_eval_expression}}) evaluated {{${body} >= {{hot.threshold}}}} as written and reported a parse error, while the same expression works in a route because placeholders are resolved before Simple is parsed. The evaluator should resolve placeholders from the running integration's properties (or from the project's application.properties when evaluating locally) before evaluating, or say clearly that a placeholder was left unresolved. Re-check after CAMEL-24692.
> h3. 4. Validator error messages should say what to do
> The YAML DSL validator returns the raw schema error. Two cases from the benchmark where the message stopped a small model and would slow a person:
> * "log: property 'loggerName' is not defined in the schema" should add "did you mean logName" (the edit-distance suggestion from CAMEL-24666 already exists in the catalog).
> * "route: property 'onException' is not defined in the schema" should add that onException is a top-level entry, a sibling of route, and show a five-line sample once CAMEL-24693 ships them.
> camel-yaml-dsl-validator already depends on camel-catalog, and the YAML DSL model (CAMEL-24362) knows which entries are top-level and which nest where, so this is a post-processing step over the networknt errors, no schema change.
> h3. Origin
> Rows 12 to 15 of the defects tally kept with the benchmark harness. Related: CAMEL-24692, CAMEL-24693, CAMEL-24694, CAMEL-24695, CAMEL-24666, CAMEL-24362.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)