[jira] [Resolved] (CAMEL-25150) camel-bindy - with quotingEscaped=true a CSV field whose value ends with a quote (He said "hi", 12") is not closed on unmarshal, so it swallows the next fields and the columns shift (regression of CAMEL-22068)

["Claus Ibsen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Claus Ibsen resolved CAMEL-25150.
---------------------------------
    Fix Version/s: 4.23.0
       Resolution: Fixed

The fix is merged on main, so it is in Camel 4.23.0:
* cb8769029093 CAMEL-25150: camel-bindy - with quotingEscaped a CSV field ending with a quote is closed again

Resolving, as the ticket was not updated when the PR was merged.

_Claude Code on behalf of Claus Ibsen_

> camel-bindy - with quotingEscaped=true a CSV field whose value ends with a quote (He said "hi", 12") is not closed on unmarshal, so it swallows the next fields and the columns shift (regression of CAMEL-22068)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25150
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25150
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-bindy
>            Reporter: shashank
>            Priority: Minor
>              Labels: regression
>             Fix For: 4.23.0
>
>
> With {{@CsvRecord(quoting = true, quotingEscaped = true)}} Bindy writes a quote inside a field as a backslash and the quote (CAMEL-7519): the value {{He said "hi"}} is written {{"He said \"hi\""}}. Unmarshal cannot read this field back when the quote character is the double quote and the value ends with an odd number of quotes:
> {noformat}
> line:      "123","He said \"hi\"","10"
> expected:  firstField=123, secondField=He said "hi", number=10
> actual:    firstField=123, secondField=He said "hi"",10  (the third column is merged into the second), number=null
> {noformat}
> The cause is the loop that CAMEL-22068 added to {{BindyCsvDataFormat.unquoteTokens}} to support RFC 4180 doubled quotes: for a token that ends with the quote, it counts the quotes in front of the final one, skipping backslashes, and treats the final quote as escaped when the count is odd. That is the RFC 4180 rule ({{""}} is one quote in the value), but it also runs when {{quotingEscaped=true}}, where the value's quote is written {{\"}}. For {{...\""}} the loop skips the backslash, counts one quote, decides that the closing quote is escaped, and keeps the field open until a later token ends with a quote. The result is silent data corruption: the value gets the separator and the following columns appended, the following fields are shifted or empty, and a typed field (number, date) that receives the wrong text fails.
> Values with an even number of trailing quotes (the value {{""foo""}} of the existing test) and values without a trailing quote are read correctly, which is why the existing tests pass. The single quote as quote character is not affected, because the loop only runs for the double quote.
> Before 4.8.8 / 4.10.5 / 4.12.0 a token ending with the quote always closed the field, and these values were read back correctly. A brute force over all values of up to 3 characters from {{a , " \ space}} in each of three columns (marshal, then unmarshal): 120 of 468 records fail today, 33 with the unquote code of 4.10.0, 33 with the fix below. The 33 remaining ones are values with an escaped quote directly in front of the separator ({{a",b}}), which already failed before CAMEL-22068 and are not a regression.
> h3. Reproduction
> {code:java}
> @CsvRecord(separator = ",", quote = "\"", quoting = true, quotingEscaped = true)
> public static class Row {
>     @DataField(pos = 1) private String firstField;
>     @DataField(pos = 2) private String secondField;
>     @DataField(pos = 3, pattern = "########.##") private BigDecimal number;
> }
> // marshal of (123, He said "hi", 10) gives "123","He said \"hi\"","10"
> // unmarshal of that line gives secondField = He said "hi"",10 and number = null
> {code}
> A unit test fails on main (three runs) for the values {{He said "hi"}}, {{12"}}, {{"}} and {{a "b" "c"}}. A small formal model (Lean 4) of the loop shows that for every value that ends with {{n}} quotes after another character the field is closed exactly when {{n}} is even, and that with the fix every quoted field without a separator in it is read back as written.
> h3. Affected versions
> The loop is in 4.8.8, 4.10.5, 4.12.0 and every later release up to main; 4.8.7 and 4.10.4 do not have it.
> h3. Proposed fix
> Apply the doubled-quote rule only when {{quotingEscaped}} is false:
> {code:java}
> if (quote.equals("\"") && !quotingEscaped) {
>     // RFC 4180 doubled quotes
>     ...
> }
> {code}
> With {{quotingEscaped=true}} a token that ends with the quote closes the field again, as before CAMEL-22068. RFC 4180 input ({{quotingEscaped=false}}, the default) is read exactly as today, including the cases of CAMEL-22068. Values that end with a backslash ({{C:\temp\}}) keep working.
> The one input that is read differently is RFC 4180 style input in {{quotingEscaped=true}} mode with a separator inside the quotes and an odd number of quotes in front of it ({{"x"",y"}}): today it stays one field (with the doubled quote kept, since this mode does not unescape {{""}}), with the fix it is split, as in 4.8.7 and 4.10.4. Such input is not the format {{quotingEscaped}} describes.
> Duplicate check (2026-09-30): JIRA "quotingEscaped", "bindy" with "escape" and "quote": CAMEL-7519, CAMEL-12274, CAMEL-22068, CAMEL-19143, all fixed, none about this case. No open pull request touches {{unquoteTokens}}.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)