You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "shashank (Jira)" <ji...@apache.org> on 2026/10/12 10:24:00 UTC
[jira] [Created] (CAMEL-25574) camel-zip-deflater - after unmarshal().gzipDeflater() the stream-cached body cannot be read again from its start: a dead letter channel with useOriginalMessage gets only the end of the original body
shashank created CAMEL-25574:
--------------------------------
Summary: camel-zip-deflater - after unmarshal().gzipDeflater() the stream-cached body cannot be read again from its start: a dead letter channel with useOriginalMessage gets only the end of the original body
Key: CAMEL-25574
URL: https://issues.apache.org/jira/browse/CAMEL-25574
Project: Camel
Issue Type: Bug
Reporter: shashank
Assignee: shashank
{{GzipDeflaterDataFormat.unmarshal}} (lines 57-72 of main df85d15a3dcb) gives the stream it gets to {{GzipCompressorInputStream}} of commons-compress. When the body of the message is an {{InputStream}}, that stream is the body itself ({{UnmarshalProcessor}}, lines 90-92), and with stream caching (on by default) the body is an in-memory stream cache: {{InputStreamCache}}, which extends {{ByteArrayInputStream}}, or {{ByteArrayInputStreamCache}}, which delegates {{mark}} and {{reset}} to a {{ByteArrayInputStream}}.
The gzip reader uses a stream with {{markSupported()}} as it is, and calls {{mark}} before every buffer (8192 bytes) that it reads, to {{reset}} and step back when the compressed data ends inside the buffer. So after the unmarshal the mark of the stream cache is behind the 10 byte gzip header, or at the start of the last buffer of a larger body. {{reset()}} of a {{ByteArrayInputStream}} goes back to the last mark, so {{StreamCache.reset()}} ("Resets the StreamCache for a new stream consumption") no longer goes back to the start of the body.
Camel resets the stream cache whenever the body is to be read again, and the original message that a unit of work keeps for {{useOriginalMessage}} holds the same stream cache object ({{DefaultUnitOfWork}}, lines 125-136). Effects, with the default configuration:
* A dead letter channel with {{useOriginalMessage()}}, in a route that fails after {{unmarshal().gzipDeflater()}} (by the same mechanism {{onException(...).useOriginalMessage()}}; not run): the dead letter gets the original body without its first bytes. For a gzip body of 31 bytes it gets the last 21, for one of 20028 bytes the last 3634. It is not a gzip stream any more, and nothing is logged.
* A route that keeps the compressed body (in a property, a variable or a header) and uses it after the unmarshal, for example to archive it, gets the same.
Not affected: a body that is a {{byte[]}} or a file (a new stream is opened for each read), and a stream cache that is spooled to disk ({{FileInputStreamCache}} does not support {{mark}}, so the reader wraps it; read in the code, not run).
h3. Reproduction
New {{GzipDataFormatBodyReadAgainTest}} (3 tests, plain {{direct}} routes, no option set). With the {{GzipDeflaterDataFormat}} of main all 3 fail (run again with the final test class, 2026-10-12):
* {{deadLetterChannelGetsTheWholeOriginalBody}} ({{errorHandler(deadLetterChannel("direct:dead").useOriginalMessage())}}, {{unmarshal().gzipDeflater()}}, then an exception; the dead letter route converts the body to {{byte[]}}): {{array lengths differ, expected: <31> but was: <21>}}
* {{deadLetterChannelGetsTheWholeOriginalBodyOfALargeMessage}}: {{array lengths differ, expected: <20028> but was: <3634>}}
* {{theCompressedBodyKeptInAPropertyCanBeReadAfterTheUnmarshal}} ({{setProperty("compressed", body())}}, unmarshal, {{setBody(exchangeProperty("compressed"))}}): {{array lengths differ, expected: <31> but was: <21>}}
h3. Proposed fix
{{unmarshal}} wraps the stream in a {{BufferedInputStream}} before it gives it to the gzip reader, so the reader marks and resets a stream of its own and the body is only read. For a stream without mark support the reader already did the same wrapping itself; for the in-memory stream caches this adds one copy through a buffer of 8192 bytes.
The unmarshalled result is the same, so there is no upgrade note. camel-zip-deflater with the fix: 16 tests, 0 failures.
Not changed: the in-memory stream caches themselves. Any reader that calls {{mark}} in the middle of a stream-cached body moves the position that {{StreamCache.reset()}} goes back to. There is one more place in the main sources of Camel that moves the mark of such a stream, and the proposed fix does not change it: {{IOHelper.copy(InputStream, OutputStream, int, boolean, long)}} ({{IOHelper.java}}, lines 224-227) calls {{mark(0)}} and {{reset()}} on any {{ByteArrayInputStream}} (so on an {{InputStreamCache}}, not on a {{ByteArrayInputStreamCache}}) to get its size, which sets the mark to the position the stream has when the copy starts. For a stream at its start (the usual case, Camel resets the stream cache before each step) nothing changes; for a stream that was already partly read, a later {{reset()}} goes back to that position and not to the start. Its callers with a stream are {{DefaultHttpBinding.copyStream}}, {{UndertowConsumer}} ({{useStreaming}}) and {{CachedOutputStream}} (a new stream); a case where one of them gets a partly read body was not run. Whether {{reset()}} of the stream caches should always go to the start is a question for a separate ticket.
Related, and independent of this ticket: the read contract of the request body stream cache of camel-netty-http ({{NettyChannelBufferStreamCache}}) has its own ticket. With the fix proposed there that stream cache does not support {{mark}}, so the gzip reader wraps it by itself and neither fix needs the other.
Found with a Lean 4 model of the stream cache (length, position, mark) and of the buffer loop of the gzip reader: the model shows that on main a {{reset()}} after the unmarshal never goes back before the end of the gzip header, for every size of body, so only the tail is returned (21 of 31 and 3634 of 20028 bytes in the two tested cases). The fix is modelled as not marking the body at all, so for the fix the model says nothing beyond that; the tests are the evidence for it. Then confirmed with the data format as above.
Affected: main; camel-4.22.x, camel-4.18.x and camel-4.14.x use the same reader on the body (read, not run). Since 3.12.0 (CAMEL-16953).
Duplicate check (2026-10-12): JIRA text "gzip useOriginalMessage" (1, unrelated), "gzipDeflater original message" (none), "GzipCompressorInputStream" (1: CAMEL-16953, the change that introduced the reader), "stream cache mark reset" (8, none about a reader moving the mark: CAMEL-5008, CAMEL-12078, CAMEL-16252 about azure-storage-blob needing mark support, CAMEL-25167, CAMEL-5940, CAMEL-24159, CAMEL-12955, CAMEL-12297), "gzip unmarshal stream cache truncated" (none); GitHub pull requests "gzipDeflater" (2 merged: #26619 YAML validation, #6864 docs), "GzipDeflaterDataFormat" (none), "zip-deflater" in the title (1 merged: #23642 deprecated methods), none open.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)