You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ivan Andika (Jira)" <ji...@apache.org> on 2024/10/17 06:03:00 UTC
[jira] [Comment Edited] (HDDS-11109) Fail MPU earlier for too small part
[ https://issues.apache.org/jira/browse/HDDS-11109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17890323#comment-17890323 ]
Ivan Andika edited comment on HDDS-11109 at 10/17/24 6:02 AM:
--------------------------------------------------------------
> UploadPart request has no information about being the last part
FYI, I think UploadPartsRequest can specify whether it is a last part, we might be able to use it.
* SDK V1: [https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/UploadPartRequest.html#isLastPart]
* SDK V2: [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/UploadPartRequest.html#sdkPartType()]
Haven't verified whether this is actually set in the SDK implementation.
was (Author: JIRAUSER298977):
> UploadPart request has no information about being the last part
FYI, I think UploadPartsRequest can specify whether it is a last part, we might be able to sue it.
* SDK V1: [https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/UploadPartRequest.html#isLastPart]
* SDK V2: [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/UploadPartRequest.html#sdkPartType()]
Haven't verified whether this is actually set in the SDK implementation.
> Fail MPU earlier for too small part
> -----------------------------------
>
> Key: HDDS-11109
> URL: https://issues.apache.org/jira/browse/HDDS-11109
> Project: Apache Ozone
> Issue Type: Improvement
> Components: s3gateway
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Major
>
> Ozone requires S3 multipart upload parts to be at least 5MB, with the exception of the last part. This requirement is enforced only while processing CompleteMultipartUpload request, because UploadPart request has no information about being the last part.
> This may be improved: when receiving any but the very first part, one of the parts can verified (either the previous one, or the current one, depending on upload order and considering part replacement).
> With normal upload order: receiving part N, we can verify part N-1 is acceptable.
> Potential downsides:
> # Use case that would no longer work: client uploads multiple undersized parts, but CompleteMultipartUpload lists only one of them, and at the last position; the rest are orphaned.
> # May not be fully compatible with AWS.
> https://github.com/apache/ozone/discussions/6869
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org