You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ivan Andika (Jira)" <ji...@apache.org> on 2023/09/19 09:59:00 UTC
[jira] [Updated] (HDDS-9318) Prevent bucket from being deleted if incomplete MPUs exist
[ https://issues.apache.org/jira/browse/HDDS-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Andika updated HDDS-9318:
------------------------------
Description:
While working on the MPU cleanup service, there are possible scenarios that might cause issues when cleaning up / abort incomplete MPUs:
# Cleaning up / abort MPUs whose bucket has been deleted
# Cleaning up / abort MPUs whose bucket has just been recreated
# Cleaning up / abort MPUs whose bucket has changed layout (i.e. delete the original bucket with OBS layout and creating an FSO bucket with the same name)
For case 2, if we abort the newly created bucket, the bucket might end up with negative usedBytes since S3MultipartUploadAbortRequest will substract from the bucket quota.
For case 3, if the bucket is change layout from OBS -> FSO, the abort MPU will fail since the MPU open key is located in the openKeyTable, but due to the new FSO bucket layout, it will check openFileTable.
Therefore, to resolve above cases, let's have an invariant that "all incomplete MPUs must belongs to an existing bucket" by disallowing bucket deletion until all the MPUs have been aborted.
was:
While working on the MPU cleanup service, there are possible scenarios that might cause issues when cleaning up incomplete MPUs:
# Cleaning up MPUs whose bucket has been deleted
# Cleaning up MPUs whose bucket has changed layout (i.e. delete the original bucket with OBS layout and creating an FSO bucket with the same name)
Therefore, let's set up an invariant that all incomplete MPUs must belongs to an existing bucket by disallowing bucket deletion until all the MPUs have been aborted.
> Prevent bucket from being deleted if incomplete MPUs exist
> ----------------------------------------------------------
>
> Key: HDDS-9318
> URL: https://issues.apache.org/jira/browse/HDDS-9318
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: OM
> Affects Versions: 1.3.0
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Fix For: 1.4.0
>
>
> While working on the MPU cleanup service, there are possible scenarios that might cause issues when cleaning up / abort incomplete MPUs:
> # Cleaning up / abort MPUs whose bucket has been deleted
> # Cleaning up / abort MPUs whose bucket has just been recreated
> # Cleaning up / abort MPUs whose bucket has changed layout (i.e. delete the original bucket with OBS layout and creating an FSO bucket with the same name)
> For case 2, if we abort the newly created bucket, the bucket might end up with negative usedBytes since S3MultipartUploadAbortRequest will substract from the bucket quota.
> For case 3, if the bucket is change layout from OBS -> FSO, the abort MPU will fail since the MPU open key is located in the openKeyTable, but due to the new FSO bucket layout, it will check openFileTable.
> Therefore, to resolve above cases, let's have an invariant that "all incomplete MPUs must belongs to an existing bucket" by disallowing bucket deletion until all the MPUs have been aborted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org