You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Claus Ibsen (Jira)" <ji...@apache.org> on 2026/07/17 15:29:00 UTC
[jira] [Commented] (CAMEL-24153) camel-aws2-s3 - Streaming upload silently truncates bodies larger than partSize and orphans multipart uploads
[ https://issues.apache.org/jira/browse/CAMEL-24153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18097122#comment-18097122 ]
Claus Ibsen commented on CAMEL-24153:
-------------------------------------
Fixed via https://github.com/apache/camel/pull/24859
> camel-aws2-s3 - Streaming upload silently truncates bodies larger than partSize and orphans multipart uploads
> -------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24153
> URL: https://issues.apache.org/jira/browse/CAMEL-24153
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-s3
> Reporter: Federico Mariani
> Assignee: Claus Ibsen
> Priority: Major
> Labels: review-2026-07
> Fix For: 4.22.0
>
> Attachments: AWS2S3StreamUploadProducerTruncationTest.java
>
>
> Found during a code review of camel-aws2-s3 on main (4.22.0-SNAPSHOT).
> In {{AWS2S3StreamUploadProducer.processWithoutTimestampGrouping}} (lines ~179-297), {{maxRead}} starts at {{partSize}} and is decremented after every read. It is reset back to {{partSize}} only on the *part-flush* branches (lines 198 and 288). When the *batch-flush* branch fires instead (line 271: {{totalSize >= batchSize}} -> {{uploadPart(state); completeUpload(state); state = null; continue;}}), {{maxRead}} stays {{0}}, so the next loop iteration calls {{AWS2S3Utils.toByteArray(is, 0)}}, which reads nothing, and the loop exits with the remainder of the input stream unread. The exchange still completes successfully with the ETag of the truncated object.
> Note this happens with the *default* configuration ({{batchSize=1000000}} < {{partSize=26214400}}): any body larger than partSize is truncated to the first part.
> Additionally, line 183 ({{maxRead -= uploadAggregate.buffer.size()}}) can drive {{maxRead}} negative when {{batchSize > bufferSize}}, dropping an entire subsequent exchange body.
> *Secondary bug in the same loop*: since CAMEL-20728 changed {{state.index++}} to {{state.index = 1}} (line 224), the guard {{if (state.index == 1)}} at line 265 no longer means "first iteration", so {{createMultipartUpload}} is invoked on *every* read iteration. Parts of a single logical upload are spread across several uploadIds; {{completeMultipartUpload}} then fails with {{InvalidPart}}, and the superseded uploads are never aborted (incomplete multipart uploads accrue storage charges indefinitely).
> *Reproducer*: attached {{AWS2S3StreamUploadProducerTruncationTest.java}} (pure unit test, mocked S3Client, goes under {{components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/stream/}}). It sends a 12288-byte body with {{partSize=8192}}, {{batchSize=4096}} and fails with:
> {noformat}
> expected: <12288> but was: <8192>
> {noformat}
> The existing IT (S3StreamUploadS3MultipartIT) only sends bodies smaller than partSize, which is why this path is untested.
> *Suggested fix*: reset {{maxRead}} after every flush (and clamp to a positive read size), and guard the upload creation with {{if (state.initResponse == null)}} instead of {{state.index == 1}}.
> _This issue was researched and filed by Claude Code (AI) on behalf of Federico Mariani (fmariani). Full findings document from the review is attached to the umbrella issues._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)