You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "rich7420 (via GitHub)" <gi...@apache.org> on 2026/06/08 09:48:08 UTC
[PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
rich7420 opened a new pull request, #10461:
URL: https://github.com/apache/ozone/pull/10461
## What changes were proposed in this pull request?
When a client completes a multipart upload using chunked transfer encoding
(no `Content-Length`, e.g. the AWS C++ SDK which sends the request with
`Expect: 100-continue`), Ozone S3 Gateway rejects the request with HTTP 400
`You must specify at least one part`, even though the request body does
contain the parts.
The root cause is in `CompleteMultipartUploadRequestUnmarshaller`, which used
`InputStream#available() == 0` to detect an empty request body. `available()`
only reports the number of bytes that can be read **without blocking**; for a
body that has not been buffered yet (chunked transfer / `Expect: 100-continue`)
it can legitimately return `0` even though the body is non-empty. As a result
the unmarshaller wrongly concluded the body was empty and raised the
"must specify at least one part" error before parsing the parts.
This PR replaces the unreliable `available()` check with a proper
empty-body detection: it wraps the stream in a `PushbackInputStream` and reads
a single byte. `read()` blocks until data arrives or EOF is reached, so:
* if it returns `-1`, the body is genuinely empty and the original clean error
is raised (preserving the behavior introduced in HDDS-11457);
* otherwise the byte is pushed back and parsing proceeds normally.
This is the only use of `InputStream#available()` in the S3 Gateway, so no
other request type is affected.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/projects/HDDS/issues/HDDS-14760
## How was this patch tested?
* New unit tests in `TestCompleteMultipartUploadRequestUnmarshaller`:
* `fromStreamWhereAvailableReturnsZero` reproduces the bug using a stream
whose `available()` always returns `0` while the body holds valid parts.
It fails before the fix and passes after it.
* `emptyBodyThrowsMustSpecifyAtLeastOnePart` verifies the empty-body error
is still returned.
* Existing unmarshaller tests (with/without namespace, concurrent parsing) and
the S3 Gateway multipart tests pass with no regression.
* Added an acceptance test `Test Multipart Upload Complete With Chunked
Transfer Encoding` in `MultipartUpload.robot` that completes an MPU through a
presigned URL with `Transfer-Encoding: chunked` (no `Content-Length`),
exercising the end-to-end path that the C++ SDK triggers.
* CI run on the fork: https://github.com/rich7420/ozone/actions/runs/27128059905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "deekshith993 (via GitHub)" <gi...@apache.org>.
deekshith993 commented on PR #10461:
URL: https://github.com/apache/ozone/pull/10461#issuecomment-5865063433
@rich7420 in which version release will this fix be available
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10461:
URL: https://github.com/apache/ozone/pull/10461#issuecomment-4794324828
Thanks @rich7420 for working on this. Overall LGTM. The production fix looks correct to me; I only have two minor, non-blocking test-hardening comments.
1. I agree with the reviewer that the Robot smoke test should explicitly verify the request was actually sent with `Transfer-Encoding: chunked` and without `Content-Length`, so it cannot silently stop covering the regression path.
2. In `checkContent()`, it would also be good to assert the parsed `PartNumber` values in addition to the ETags. For example, checking `1` and `2` would catch a parsing regression where part numbers are dropped or swapped.
Both are minor and non-blocking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #10461:
URL: https://github.com/apache/ozone/pull/10461#issuecomment-4654037036
Thank you for the fix. Review incoming.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10461:
URL: https://github.com/apache/ozone/pull/10461#issuecomment-4826688855
The changes look good to me. LGTM!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on code in PR #10461:
URL: https://github.com/apache/ozone/pull/10461#discussion_r3482182028
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequestUnmarshaller.java:
##########
@@ -50,11 +51,18 @@ public CompleteMultipartUploadRequest readFrom(
MultivaluedMap<String, String> multivaluedMap,
InputStream inputStream) throws WebApplicationException {
try {
- if (inputStream.available() == 0) {
+ // Detect an empty request body by trying to read a single byte rather
+ // than relying on InputStream#available(), which may return 0 even when
+ // the body is not yet buffered (e.g. with Expect: 100-continue). See
+ // HDDS-14760.
+ PushbackInputStream pushbackStream = new PushbackInputStream(inputStream);
Review Comment:
btw, thanks for your asking
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "Russole (via GitHub)" <gi...@apache.org>.
Russole commented on PR #10461:
URL: https://github.com/apache/ozone/pull/10461#issuecomment-4845377198
Hi @kerneltime, just wanted to check if you’d still like to review this or share any feedback. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on code in PR #10461:
URL: https://github.com/apache/ozone/pull/10461#discussion_r3482180387
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequestUnmarshaller.java:
##########
@@ -50,11 +51,18 @@ public CompleteMultipartUploadRequest readFrom(
MultivaluedMap<String, String> multivaluedMap,
InputStream inputStream) throws WebApplicationException {
try {
- if (inputStream.available() == 0) {
+ // Detect an empty request body by trying to read a single byte rather
+ // than relying on InputStream#available(), which may return 0 even when
+ // the body is not yet buffered (e.g. with Expect: 100-continue). See
+ // HDDS-14760.
+ PushbackInputStream pushbackStream = new PushbackInputStream(inputStream);
Review Comment:
The performance impact should be negligible. This adds one small PushbackInputStream wrapper and performs a single-byte read/unread before handing the same stream to the XML parser. The parser would have to perform that first read immediately anyway, so I think this does not add meaningful I/O or buffering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "yandrey321 (via GitHub)" <gi...@apache.org>.
yandrey321 commented on code in PR #10461:
URL: https://github.com/apache/ozone/pull/10461#discussion_r3470809825
##########
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/CompleteMultipartUploadRequestUnmarshaller.java:
##########
@@ -50,11 +51,18 @@ public CompleteMultipartUploadRequest readFrom(
MultivaluedMap<String, String> multivaluedMap,
InputStream inputStream) throws WebApplicationException {
try {
- if (inputStream.available() == 0) {
+ // Detect an empty request body by trying to read a single byte rather
+ // than relying on InputStream#available(), which may return 0 even when
+ // the body is not yet buffered (e.g. with Expect: 100-continue). See
+ // HDDS-14760.
+ PushbackInputStream pushbackStream = new PushbackInputStream(inputStream);
Review Comment:
what would be the performance implication of this logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14760. Fix CompleteMultipartUpload rejecting chunked request body as empty [ozone]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #10461:
URL: https://github.com/apache/ozone/pull/10461#discussion_r3376698031
##########
hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestCompleteMultipartUploadRequestUnmarshaller.java:
##########
@@ -58,6 +63,44 @@ public void fromStreamWithNamespace() throws IOException {
checkContent(completeMultipartUploadRequest);
}
+ @Test
+ public void fromStreamWhereAvailableReturnsZero() throws IOException {
+ // Simulates a request body stream (e.g. with Expect: 100-continue) where
+ // InputStream#available() returns 0 even though the body has not been
+ // fully buffered yet. See HDDS-14760.
+ InputStream inputBody = new FilterInputStream(new ByteArrayInputStream(
+ ("<CompleteMultipartUpload xmlns=\"" + S3Consts.S3_XML_NAMESPACE + "\">" +
+ "<Part><ETag>" + part1 + "</ETag><PartNumber>1" +
+ "</PartNumber></Part><Part><ETag>" + part2 +
+ "</ETag><PartNumber>2</PartNumber></Part>" +
+ "</CompleteMultipartUpload>").getBytes(UTF_8))) {
+ @Override
+ public int available() {
+ return 0;
+ }
+ };
+
+ //WHEN
+ CompleteMultipartUploadRequest completeMultipartUploadRequest =
+ new CompleteMultipartUploadRequestUnmarshaller()
+ .readFrom(null, null, null, null, null, inputBody);
+
+ //THEN
+ checkContent(completeMultipartUploadRequest);
+ }
+
+ @Test
+ public void emptyBodyThrowsMustSpecifyAtLeastOnePart() {
+ InputStream emptyBody = new ByteArrayInputStream(new byte[0]);
+
+ WebApplicationException ex = assertThrows(WebApplicationException.class,
+ () -> new CompleteMultipartUploadRequestUnmarshaller()
+ .readFrom(null, null, null, null, null, emptyBody));
+
+ assertTrue(ex.getMessage().contains("must specify at least one part"),
+ "Unexpected message: " + ex.getMessage());
Review Comment:
This test asserts on `WebApplicationException#getMessage()`, which is more fragile than asserting on the underlying `OS3Exception` returned as the cause (that’s what `wrapOS3Exception` sets). Asserting on the cause makes the test resilient to changes in how the JAX-RS exception formats its message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org