You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Gargi-jais11 (via GitHub)" <gi...@apache.org> on 2026/07/24 09:08:37 UTC
[PR] HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject [ozone]
Gargi-jais11 opened a new pull request, #10860:
URL: https://github.com/apache/ozone/pull/10860
## What changes were proposed in this pull request?
S3 Gateway returns `404 NoSuchKey` for GetObject requests with unreadable/invalid object keys, instead of `400 InvalidURI `as AWS S3 does.
Ozone already defines the correct error in S3ErrorTable.INVALID_URI but never uses it:
`INVALID_URI("InvalidURI", "Couldn't parse the specified URI.", HTTP_BAD_REQUEST)`
The key **\xae\x8a**- contains non-printable high bytes (\x80–\xff) that Ozone cannot store and that AWS treats as an invalid URI. S3G should reject it before OM lookup.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15958
## How was this patch tested?
Add Unit and Integration Tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #10860:
URL: https://github.com/apache/ozone/pull/10860#issuecomment-5113275410
Thanks @ayushtkn for reviewing the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on PR #10860:
URL: https://github.com/apache/ozone/pull/10860#issuecomment-5087752564
> I see that this logic was likely added based on the AWS S3 docs mentioning `'Non-printable ASCII characters (128–255 decimal characters)'` under the `'Characters to avoid'` section.
>
> However, there are two issues with enforcing it this way: AWS S3 doesn't actually block these: The AWS docs list this under `'Characters to avoid'` as a recommendation for clients to prevent application-side parsing issues.
>
> AWS S3 itself still accepts valid `UTF-8` characters that resolve to this range; it does not throw an InvalidURI error.
>
> This blocks valid international characters: Because I believe `keyPath` is a Java String (`UTF-16`), checking `c >= 0x80` && `c <= 0xFF` targets the `Unicode Latin-1 Supplement` block. This will hard-reject perfectly valid, printable characters like `é `(`U+00E9 / 233`), `ñ` (`U+00F1 / 241`), or `£` (`U+00A3 / 163`).
>
> A user uploading a file named `café.txt` or `piñata.jpg` will get an unexpected `InvalidURI` error.
>
> The Ceph `test_object_read_unreadable` test is likely sending raw, unencoded invalid bytes over the wire to trigger a parsing failure. We should rely on the `\uFFFD` check to catch malformed `UTF-8`, and if we want to block actual control characters, we should use `Character.isISOControl(c)` instead of blocking the entire `128-255` block.
>
> https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines
>
> Or Am I missing somethign here?
Thanks @ayushtkn for the review. You're right — the AWS "characters to avoid (128–255)" guidance is a recommendation, not something S3 enforces server-side, and blocking that range would incorrectly reject valid UTF-8 object names like café.txt. I didn't think about this, really a good catch.
I've updated the validation to:
- Reject keys containing \uFFFD (malformed UTF-8 after URI decoding)
- Reject keys containing ISO control characters via Character.isISOControl() — this covers the s3-test case (U+008A in \xae\x8a-)
- Removed the blanket 0x80–0xFF check and added a unit test confirming valid Unicode keys like café.txt are accepted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15958. Return InvalidURI for unreadable S3 object keys on GetObject [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 merged PR #10860:
URL: https://github.com/apache/ozone/pull/10860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org