[jira] [Commented] (CAMEL-24961) camel-catalog: the body format and Java carrier each component, data format, language and EIP consumes and produces

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-24961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129596#comment-18129596 ] 

Claus Ibsen commented on CAMEL-24961:
-------------------------------------

First slice in review: https://github.com/apache/camel/pull/28241

*The shape*, from the design note on CAMEL-24844 (2026-10-05):
* A {{@BodyTypes}} annotation on the endpoint, data format or language class has one entry per side: consumerOut, consumerReply, producerIn, producerOut, marshalIn/Out, unmarshalIn/Out, expressionIn/Out.
* Each side has its carriers: null, text, bytes, stream, file, value, map, list, pojo; or unchanged, any, unknown. It can also have the Java type, the format, the options it depends on, the body for other option values ({{byOption}}) and a description.
* The catalog JSON has a {{bodyTypes}} section next to {{headers}}. {{camel_catalog_doc}} returns it.

*Changes to the design note*:
* consumerReply was added: the body a route that answers its caller must end with, as a platform-http route does.
* There is no inheritance: protobufJackson extends the jackson data format, but its payload is not JSON. Each class declares its own.
* The build rejects a side that does not fit the kind, an option name that does not exist, unchanged on an input and any on an output.

*The first slice*: file, direct, seda, timer, cron, kamelet, http/https, rest-openapi, xslt, sql and the json data format. Each value is read from the code that sets the body, with the file and line in the PR.

*Checked at runtime*: the tests of 15 modules ran with message history forced on, without a test changed, and produced 971 route coverage dumps. Reference routes covered the core components, whose tests do not dump the coverage. Every recorded body type fits the declared carriers.

*Next*: the rest of the pilot set, already read and checked: platform-http, rest, ftp/sftp, jms and its family, kafka, aws2-sqs, paho-mqtt5, mongodb, log, mock, csv, jacksonXml, yaml and the languages.

Reading the code also found three bugs, filed as CAMEL-25923 (unmarshal with allowNullBody drops the headers), CAMEL-25924 (kafka splits a Jackson ObjectNode into its field values) and CAMEL-25925 (aws2-sqs batches over 10 entries keep only the last response).

_Claude Code on behalf of davsclaus_

> camel-catalog: the body format and Java carrier each component, data format, language and EIP consumes and produces
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-24961
>                 URL: https://issues.apache.org/jira/browse/CAMEL-24961
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-catalog
>            Reporter: Claus Ibsen
>            Assignee: Claus Ibsen
>            Priority: Major
>             Fix For: 4.24.0
>
>
> Phase C of CAMEL-24844. CAMEL-11132 (2017) is the original ask and stays the umbrella; this issue is the executable spec for it.
> The catalog describes every option and every header of a component - {{file}} carries 17 headers with a javaType - and says nothing about the body. So a person, an IDE, the camel-jbang-mcp catalog tools and the YAML validator cannot answer the question that is asked most: *what is the body after this step*. The validator's body-presence walk (CAMEL-24844 phase A) has to carry its own hardcoded rules for that reason, which is the wrong place for them.
> h3. What to add
> Per component, data format, language and EIP, in the JSON the catalog already generates, two dimensions:
> * the *format* - what the payload is, with the Kamelet data type names so a Kamelet and a plain component read the same: json, xml, csv, text, binary, avro...
> * the *carrier* - the Java type the format arrives in: text, bytes, stream, file, map, list, pojo, null. This is what the failures are about: a JSON body as a {{Map}} and as text look identical in a description and behave differently in {{${body.orderId}}}, {{jsonpath}} and Groovy.
> Both for input and output, and where the type depends on an option ({{unmarshalType}}, {{charset}}, stream caching) the metadata names the option rather than picking one value.
> h3. Every entry is truthful: four values, no guesses
> A half-filled field is worse than an empty one - a consumer cannot tell "unknowable" from "not done yet". So all 407 components get an entry, and the value is one of:
> # *derived* - read from the component's own source at its {{setBody}} call site, recorded with file and line as provenance so a reviewer checks it in seconds. Measured: *322 of 407* components have a {{setBody}} call site in their package, and the argument is nearly always a local or field whose declared type is in the same file ({{result}} 519, {{response}} 191, {{body}} 191, {{out}} 53). This is a reading task, not a guessing task.
> # *unchanged* - the component does not replace the body ({{browse}}, {{bean-validator}}, {{crypto}} which writes a header). A fact, not an absence.
> # *same as <scheme>* - delegating components state the reference instead of repeating it: {{activemq}}, {{activemq6}}, {{amqp}} to jms; {{class}} to bean; {{cron}} to quartz; the {{debezium-*}} family to debezium-common.
> # *unknown* - the genuinely dynamic ones, named as such. This is the case CAMEL-11132 describes: a component whose type comes from its configuration or from the remote system's schema.
> The 85 components with no {{setBody}} in their package fall almost entirely into 2 and 3, so *unknown* should end up a small set rather than the residue.
> h3. How it is verified
> # source-derived, with the citation, for the 322;
> # the message history of the reference runs (camel-jbang-examples and the doc examples) records the body class per step, so a derived carrier that disagrees with what actually flowed fails the check - this is what catches a systematic error in the derivation itself, and it is the runtime half of CAMEL-24844 (merged in 4.23.0) earning its keep;
> # a reviewer pass over the entries that are neither derived nor confirmed.
> h3. Scope note
> The routes people actually write concentrate hard - over 1712 real YAML files (camel-jbang-examples, camel-examples, camel-kamelets, benchmark output) there are 122 distinct schemes and the top 10 cover 89.5% of uses, with {{json}} at 1214 of 1317 data format uses. That concentration says where to *start* and where the benchmark can measure an effect. It does not say where to stop: the catalog is a public contract and is filled completely.
> h3. What it unlocks
> * the validator carries the type from step to step and suggests the step that fixes it - {{convertBodyTo}}, {{unmarshal}}, {{transformDataType}} - instead of only reporting (CAMEL-24844 phase D);
> * the catalog tools and the component documentation answer "what is the body here" from metadata rather than prose;
> * the rules leave the validator's source and become data every tool shares.
> Prior art to reuse rather than invent: the Kamelet spec already has {{dataTypes:}} for in, out and error (19 of 250 Kamelets declare it today), and CAMEL-18698 shipped the data type transformer registry and the {{transformDataType}} EIP in 4.0 as the conversion mechanism.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)