[jira] [Resolved] (CAMEL-25722) camel-flatpack - marshal drops the first row of the List

["Claus Ibsen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Claus Ibsen resolved CAMEL-25722.
---------------------------------
    Resolution: Fixed

Merged in https://github.com/apache/camel/pull/28030 for 4.24.0.

_Claude Code on behalf of Claus Ibsen_

> camel-flatpack - marshal drops the first row of the List<Map> body with the default ignoreFirstRecord=true, so the first record is silently lost and unmarshal followed by marshal does not round trip
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25722
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25722
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-flatpack
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>             Fix For: 4.24.0
>
>
> {{FlatpackDataFormat.marshal}} (main 9fb9bc911f78) writes the header line with {{writer.printHeader()}} and then loops over the maps of the body, but skips the first map when {{ignoreFirstRecord}} is true, which is the default:
> {code:java}
> writer.printHeader();
> for (Map<String, Object> row : data) {
>     if (ignoreFirstRecord && first) {
>         // skip first row
>         first = false;
>         continue;
>     }
>     ...
> {code}
> The header line does not come from the first map: without a definition the column names are the keys of the first map, with a definition they come from the pzmap file. So the skipped map is a data row. Marshalling {{[ {ID=1, NAME=alice}, {ID=2, NAME=bob} ]}} with a default {{new FlatpackDataFormat()}} gives:
> {noformat}
> ID,NAME
> 2,bob
> {noformat}
> There is no error and no log. {{unmarshal}} returns a {{DataSetList}} without the header line, so {{unmarshal().flatpack()}} followed by {{marshal().flatpack()}} loses the first record of every file. The option is documented as "Whether the first line is ignored for delimited files (for the column headers)", which is what unmarshal does; nothing documents that marshal drops a data row. The component page shows a marshal example that sets {{ignoreFirstRecord(false)}} for a fixed-length format, without saying why.
> Trigger: default options; {{marshal().flatpack()}} (delimited without or with a definition, or fixed-length with a definition) of a {{List<Map<String, Object>>}}.
> h3. Reproduction
> Found by an executed round-trip property (random rows, 400 cases: every case lost its first row). New test {{FlatpackMarshalKeepsFirstRowTest}} (3 tests), all fail on main:
> * default data format, 2 maps: expected {{[ID,NAME, 1,alice, 2,bob]}}, main {{[ID,NAME, 2,bob]}};
> * data format with the {{INVENTORY-Delimited.pzmap.xml}} definition, 2 maps: expected 3 lines, main 2;
> * unmarshal of {{ID,NAME / 1,alice / 2,bob / 3,carol}} then marshal: main {{[ID,NAME, 2,bob, 3,carol]}}.
> h3. Proposed fix
> Marshal writes every map of the list; {{ignoreFirstRecord}} keeps its meaning for unmarshal. With the fix the camel-flatpack tests pass: 41 run, 0 failures, 0 errors, 0 skipped. A route that added a dummy first map to work around the defect now gets that map in the output; a note goes into the 4.24 upgrade guide.
> Affected: main (tested); camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x have the same lines (read, not run).
> Duplicate check (2026-10-17): JIRA text "flatpack marshal", "flatpack first row", "flatpack ignoreFirstRecord", camel-flatpack issues since 2024: only CAMEL-24170 (item M18 lists this defect as Medium; the two PRs that resolved the umbrella, #24946 and #24952, do not touch Flatpack). GitHub pull requests "flatpack": none about this.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)