You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ivan Andika (Jira)" <ji...@apache.org> on 2023/07/31 10:00:01 UTC
[jira] [Updated] (HDDS-9098) Exclude MPU-related open keys in OpenKeyCleanupService
[ https://issues.apache.org/jira/browse/HDDS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Andika updated HDDS-9098:
------------------------------
Description:
Currently OpenKeyCleanupService deletes all the open keys in the openKeyTable regardless whether the open keys are MPU-related or not. This might cause orphan MPU in MultipartInfoTable, since to abort the MultipartInfo, we need to check openKeyTable for MPU-related open keys, which will fail if the open keys are already deleted.
We need to exclude the MPU-related open keys in OpenKeyCleanupService. There are few considerations:
# HDDS-9017 set the isMultipartKey in these MPU-related open keys, and we can use this flag to exclude these keys from deletion
# For existing MPU-related keys prior to HDDS-9017, we can exploit the unique characteristics of the MPU-related open keys:
## Open MPU-related open keys' DB keys have certain patterns
### Open MPU Key Info (dbKey = /\{vol}/\{bucket}/\{key}{*}/\{uploadId}{*}) where uploadId = "UUID.randomUUID().toString() + "-" + UniqueId.next()"
### Open MPU Part Key Info (dbKey = /\{vol}/\{bucket}/\{key}{*}/\{clientId}{*}) where clientId = UniqueId#next (This is also the pattern of normal MPU keys have, but we might not need since S3 abort request do not check against the existence of this key)
## Open MPU-related open keys do not store any block information (unlike the normal non-MPU open keys)
### We can add this check on top of the previous check
> Exclude MPU-related open keys in OpenKeyCleanupService
> ------------------------------------------------------
>
> Key: HDDS-9098
> URL: https://issues.apache.org/jira/browse/HDDS-9098
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Currently OpenKeyCleanupService deletes all the open keys in the openKeyTable regardless whether the open keys are MPU-related or not. This might cause orphan MPU in MultipartInfoTable, since to abort the MultipartInfo, we need to check openKeyTable for MPU-related open keys, which will fail if the open keys are already deleted.
> We need to exclude the MPU-related open keys in OpenKeyCleanupService. There are few considerations:
> # HDDS-9017 set the isMultipartKey in these MPU-related open keys, and we can use this flag to exclude these keys from deletion
> # For existing MPU-related keys prior to HDDS-9017, we can exploit the unique characteristics of the MPU-related open keys:
> ## Open MPU-related open keys' DB keys have certain patterns
> ### Open MPU Key Info (dbKey = /\{vol}/\{bucket}/\{key}{*}/\{uploadId}{*}) where uploadId = "UUID.randomUUID().toString() + "-" + UniqueId.next()"
> ### Open MPU Part Key Info (dbKey = /\{vol}/\{bucket}/\{key}{*}/\{clientId}{*}) where clientId = UniqueId#next (This is also the pattern of normal MPU keys have, but we might not need since S3 abort request do not check against the existence of this key)
> ## Open MPU-related open keys do not store any block information (unlike the normal non-MPU open keys)
> ### We can add this check on top of the previous check
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org