You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Claus Ibsen (Jira)" <ji...@apache.org> on 2026/07/20 17:13:00 UTC
[jira] [Resolved] (CAMEL-24170) camel dataformats - Umbrella: remaining high/medium findings from the July 2026 data format review
[ https://issues.apache.org/jira/browse/CAMEL-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-24170.
---------------------------------
Resolution: Fixed
> camel dataformats - Umbrella: remaining high/medium findings from the July 2026 data format review
> --------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24170
> URL: https://issues.apache.org/jira/browse/CAMEL-24170
> Project: Camel
> Issue Type: Bug
> Reporter: Federico Mariani
> Assignee: Claus Ibsen
> Priority: Major
> Labels: ai-assisted-review, dataformat
> Fix For: 4.22.0
>
> Attachments: camel-dataformats-review-2026-07-public.md
>
>
> This umbrella collects the remaining findings from a July 2026 review of the Camel *data format* components (JSON: Jackson/JacksonXML + Jackson3 twins, Gson, Fastjson, Jsonb, CBOR; XML: JAXB, SOAP; CSV: commons-csv, univocity-parsers; fixed/flat: Bindy, Flatpack; YAML: SnakeYAML; binary: Avro, Protobuf, Jackson-Avro/Protobuf; archives: zipfile, tarfile; compression/encoding: base64, lzf, zip-deflater), plus the DSL model -> reifier -> data-format option wiring.
> The critical finding and the eight highest-impact highs are filed separately with attached failing reproducers: C1=CAMEL-24152, H1=CAMEL-24162, H3=CAMEL-24163, H4=CAMEL-24164, H5=CAMEL-24165, H7=CAMEL-24166, H8=CAMEL-24167, H12=CAMEL-24168, H14=CAMEL-24169. The full findings document (with all file:line references, root-cause traces and the medium/low tail) is attached to this issue as {{camel-dataformats-review-2026-07-public.md}}.
> A recurring theme: commit {{5dfc5c2857e9}} (CAMEL-22354, "align data formats getters/setters to model", released 4.15.0) is the root cause of several regressions here (the SnakeYAML critical, the CSV {{format}} regression, the YAML {{typeFilter}} builder bug, and the SOAP {{ignoreUnmarshalledHeaders}} gaps). A focused audit of everything that commit touched is worthwhile.
> h2. Remaining HIGH findings (not filed individually)
> * *H2 - camel-jacksonxml {xmlMapper} DSL option requires an undocumented {#} prefix.* {JacksonXMLDataFormatReifier} does {properties.put("xmlMapper", definition.getXmlMapper())} without {asRef()} (contrast {JsonDataFormatReifier} for {objectMapper}). {<jacksonXml xmlMapper="myXmlMapper"/>} fails with {NoTypeConversionAvailableException: String -> XmlMapper}; only {"#myXmlMapper"} works. Same reifier serves camel-jackson3xml. Runtime-reproduced.
> * *H9 - camel-univocity-parsers {lazyLoad=true} + {asMap=true} uses a {static ThreadLocal} for headers.* {HeaderRowProcessor} stores captured headers in a static ThreadLocal; consuming the lazy iterator on another thread (parallel split, seda/kafka hand-off) NPEs, and a pooled thread can carry stale headers from a different exchange, silently mapping values to wrong keys. Runtime-reproduced (cross-thread NPE).
> * *H10 - camel-univocity-parsers {lazyLoad=true} leaks the input stream* when the iterator is not fully consumed. No on-completion hook (camel-csv fixed the identical problem via {addOnCompletion}, CAMEL-15105). Abandoned split / partial read / downstream error leaks the reader/FileInputStream.
> * *H11 - camel-bindy KVP unmarshal truncates any value containing the key-value separator.* {BindyKeyValuePairFactory} does {s.split(getKeyValuePairSeparator())} (no limit) and keeps only element [1]; {58=a=b} -> {a}. Fix: {split(sep, 2)}. Runtime-reproduced.
> * *H13 - camel-bindy fixed-length {UnicodeHelper.indexOf} off-by-one.* Loop {index + len < length()} should be {<=}; a delimiter at end-of-record is unreachable, then {substring(0, -1)} -> {IndexOutOfBoundsException}. Runtime-reproduced.
> * *H15 - camel-protobuf JSON unmarshal uses the platform default charset* while marshal writes UTF-8 ({new InputStreamReader(inputStream)} vs {StandardCharsets.UTF_8}). Asymmetric round-trip on non-UTF-8 JVMs. Fix: pass UTF-8.
> * *H16 - camel-avro {contentTypeHeader} option silently ignored.* The model {AvroDataFormat} declares the attribute (default true) but does not implement {ContentTypeHeaderAware}, so the generic reifier never transfers it; {<avro contentTypeHeader="false"/>} is dead for the default {avroJackson} library. Fix: implement {ContentTypeHeaderAware}.
> * *H17 - camel-core YAMLDataFormat.Builder.typeFilter(Class...) joins names with "." instead of ",".* {typeFilter(Foo.class, Bar.class)} yields one bogus pattern {com.x.Foo.com.y.Bar} matching neither class -> every unmarshal throws. Fails closed. Fix: {new StringJoiner(",")}. Runtime-reproduced. (CAMEL-22354.)
> h2. MEDIUM findings
> _(One additional medium-severity security-hardening item identified in this review is intentionally omitted from this public issue and is being handled through the private ASF security process per SECURITY.md.)_
> * *M2 - Gson unmarshal InputStream fallback ignores {Exchange.CHARSET_NAME}* ({new InputStreamReader(is)}) while marshal honors it. Same class also in JAXB filtered-unmarshal fallback and Jsonb/Fastjson.
> * *M3 - Jsonb never sets Content-Type; {contentTypeHeader} is a no-op* (does not implement {DataFormatContentTypeHeader}).
> * *M4 - Jsonb {Jsonb} instance (AutoCloseable) never closed on stop* (doStop is a no-op) - resource leak on restart.
> * *M5 - {timezone} silently ignored for avroJackson and protobufJackson* (neither reifier Jackson branch transfers it; JSON does).
> * *M6 - {avro()} fluent Builder default library is ApacheAvro while the model default is Jackson* (CAMEL-20502 leftover) - same logical config resolves different implementations per DSL.
> * *M7 - Avro/Protobuf schema resolvers use {Class.forName} instead of the Camel class resolver* - breaks in Spring Boot fat-jar / Quarkus / OSGi.
> * *M8 - camel-avro marshalling a {GenericData.Record} without a configured schema throws* instead of using the record's own {GenericContainer} schema.
> * *M9 - SOAP fault unmarshalling NPE / UnsupportedOperationException* when the fault detail is unknown or a SOAP 1.2 {<Reason>} is missing (peer-controlled input; no null-guard, lookup outside try/catch).
> * *M10 - JAXB {filterNonXmlChars} corrupts valid supplementary-plane characters* (emoji, CJK-ext) - treats surrogate pairs as invalid, replaces with two spaces.
> * *M11 - CBOR marshal buffers the whole output in memory* ({writeValueAsBytes} then copy) instead of {writeValue(stream, graph)}.
> * *M12 - univocity fixed-width {recordEndsOnNewline=false} still enables the feature* (hardcoded {setRecordEndsOnNewline(true)}).
> * *M13 - camel-csv {captureHeaderRecord=true} ignored with {lazyLoad=true}*, and the header row is consumed and lost (only the bulk unmarshaller sets the header).
> * *M14 - camel-bindy CSV blank lines manufacture phantom empty records* (or fail if a field is required), unlike KVP which skips blank lines. Runtime-reproduced.
> * *M15 - camel-bindy CSV {defaultValue} not applied when exactly one trailing field is missing* (off-by-one {pos < totalFields}); {setDefaultValuesForFields} also NPEs on gapped positions. Runtime-reproduced.
> * *M16 - camel-bindy CSV marshal of a null {@OneToMany} list drops the child columns* (dead code passes {Field.class} as the element type). Runtime-reproduced.
> * *M17 - camel-bindy {@CsvRecord(allowEmptyStream=true)} discards real data when {InputStream.available()==0}* (a legal return for non-empty streams); two different {allowEmptyStream} flags. Runtime-reproduced.
> * *M18 - camel-flatpack marshal drops the first data row with the default {ignoreFirstRecord=true}* ({printHeader()} already emits headers; unmarshal->marshal round trip loses record #1).
> h2. LOW findings
> See the attached document for the low-severity tail (base64 urlSafe CRLF chunking and decode-side {lineSeparator} validation; Flatpack {getDataFormatName()} typo {"flatback"}; Bindy {DefaultFactoryRegistry} HashMap hot-path race and other Bindy quoting/offset/locale nits; camel-csv bulk reader leak on parser-construct failure; {useOrderedMaps}+{ignoreHeaderCase} ordering; fixed-width reifier {Integer.parseInt("null")}; jackson {useList} clobbering {collectionType}; jackson3 type-converter module-loss and dropped {canSerialize} guard; jackson {enableJacksonTypeConverter} dead property; non-mandatory {unmarshalTypeName} resolution; tarfile-vs-zipfile twin drift - empty-tar throw, {hasNext()} non-idempotence dropping zero-byte entries, marshal NPE on {CamelFileName="/"}; Avro resolver unconditional {URLDecoder.decode} and dead {validate} option; JAXB WARN-level payload logging; SnakeYAML lifecycle/type-filter widening; zip-deflater compressionLevel late validation).
> h2. Verified NON-issues (no action)
> JAXB XXE (all unmarshal paths use the hardened {StaxConverter}); fastjson2 autotype off by default; jackson3 missing {BLOCK_UNSAFE_POLYMORPHIC_BASE_TYPES} (constant removed upstream); zip-slip ({jailStartingDirectory=true}); SnakeYAML default {SafeConstructor} posture and billion-laughs limits.
> ----
> _Reported by Claude Code on behalf of Federico Mariani (Croway). Full findings document attached._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)