[jira] [Updated] (CODEC-348) Base64 STRICT decoding in 1.23.0 rejects valid input when the codec has a line length (breaking change from 1.22.1)

["Dan Mclaughlin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CODEC-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dan Mclaughlin updated CODEC-348:
---------------------------------
    Attachment: commons-codec-1.23-strict-repro.zip

> Base64 STRICT decoding in 1.23.0 rejects valid input when the codec has a line length (breaking change from 1.22.1)
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: CODEC-348
>                 URL: https://issues.apache.org/jira/browse/CODEC-348
>             Project: Commons Codec
>          Issue Type: Bug
>    Affects Versions: 1.23.0
>         Environment: OpenJDK 21.0.12
>            Reporter: Dan Mclaughlin
>            Priority: Major
>         Attachments: commons-codec-1.23-strict-repro.zip
>
>
> Component: {{org.apache.commons.codec.binary.Base64}} / {{BaseNCodec}}
> Works in: 1.22.1 and earlier (STRICT since 1.15)
> h2. Summary
> Commit {{8b506a1c}} ("Base32 and Base64 STRICT decoding now require the encoder's canonical alphabet, padding, and line separators, and validate streams through EOF") changed what {{CodecPolicy.STRICT}} means for decoding. A STRICT {{Base64}} instance constructed with a line length now accepts only byte-for-byte what that instance would encode, including a trailing line separator. Previously valid input that decoded correctly in 1.22.1 now throws {{IllegalArgumentException}}.
> This is a behavior change in a minor release that breaks existing callers who use one STRICT instance for both encoding and decoding. Shibboleth's {{Base64Support}} (used by the Shibboleth IdP, SP integrations and OpenSAML for all SAML/XML-DSig base64) is one such caller: with 1.23.0 it can no longer decode its own encoded output. This is tracked downstream as Shibboleth [JSSH-84|https://shibboleth.atlassian.net/browse/JSSH-84] (priority Highest), where it breaks SAML message, signature and certificate decoding; in our deployment it broke SAML metadata signature validation. Because the IdP does not control which commons-codec version deployers' builds resolve, every Shibboleth deployment whose dependency management moves to 1.23.0 is affected.
> h2. Steps to reproduce
> Attached {{CodecOnlyRepro.java}} (commons-codec only):
> {code:java}
> Base64 strict76 = new Base64(76, new byte[] {'\n'}, false, CodecPolicy.STRICT);
> String exact = strict76.encodeToString(data);
> strict76.decode(exact);                          // exact encoder output
> strict76.decode(exact.trim());                   // same, trailing '\n' removed
> strict76.decode(singleLineEncodingOf(data));     // same bytes, no line breaks
> strict76.decode(exact.replace("\n", "\r\n"));    // same bytes, CRLF separators
> {code}
> h2. Expected
> All four decode to the original bytes, as in 1.22.1. Each input is valid base64 for the same data, with canonical alphabet, padding and trailing bits.
> h2. Actual (1.23.0)
> ||Input||20 bytes||256 bytes||
> |Exact encoder output|OK|OK|
> |Encoder output, trailing {{\n}} trimmed|Missing or incomplete final line separator|Missing or incomplete final line separator|
> |Single line, no separators|Missing or incomplete final line separator|Invalid line separator or line length|
> |76-char lines, CRLF|Unexpected character or data after padding|Invalid line separator or line length|
> In 1.22.1 every case returns the correct bytes.
> h2. Why this is a compatibility problem
> * Through 1.22.1, {{BaseNCodec.isStrictDecoding()}} documented STRICT narrowly: decoding raises {{IllegalArgumentException}} "if trailing bits are not part of a valid encoding." Callers chose STRICT for that guarantee, not for line-structure validation.
> * {{CodecPolicy.STRICT}}'s Javadoc is unchanged in 1.23.0 ("Data that causes a codec to fail should throw an exception"), so nothing at the API level signals the broader meaning.
> * The changelog entry lists it under fixed bugs, and the release notes describe 1.23.0 as a "feature and maintenance release," so downstream consumers and automated update tools treat it as a safe minor bump.
> * Requiring a trailing separator even for input shorter than one line is stricter than RFC 2045 and RFC 4648 practice, and rejects the common pattern of trimming encoded output.
> * In CODEC-346 (resolved 2026-10-03), a request to make default decoding stricter was declined because "changing the default would introduce another compatibility break." Changing what STRICT itself accepts is the same kind of break for callers who already opted into STRICT.
> h2. Requests (any one would resolve it)
> # Make line-structure validation opt-in (for example a builder option such as {{setStrictLineValidation(true)}}), keeping 1.22.x STRICT semantics as the default.
> # Apply line-structure validation only when explicitly requested, and continue to accept input without separators (or with a missing final separator) for instances that have a line length.
> # If the new behavior stays, document it as a breaking change in the release notes and in the {{CodecPolicy.STRICT}} and {{isStrictDecoding()}} Javadoc, and state that an instance with a line length should not be used to decode arbitrary input.
> h2. Workaround for affected users
> Decode with an instance that has no line length, after removing whitespace:
> {code:java}
> new Base64(0, new byte[] {'\n'}, false, CodecPolicy.STRICT).decode(input.replaceAll("\\s+", ""));
> {code}
> This decodes all the inputs above in both 1.22.1 and 1.23.0, and still rejects non-canonical trailing bits.
> h2. Attachments
> {{commons-codec-1.23-strict-repro}} project: {{CodecOnlyRepro.java}}, {{run.sh}}, {{run-output.txt}} (output for 1.22.1 and 1.23.0). Also includes the Shibboleth {{Base64Support}} and OpenSAML reproductions showing downstream impact.
> Related: Shibboleth [JSSH-84|https://shibboleth.atlassian.net/browse/JSSH-84] (downstream impact), CODEC-346 (compatibility rationale), commit {{8b506a1c}} (the change).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)