You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "mingchao zhao (Jira)" <ji...@apache.org> on 2022/04/06 08:53:00 UTC
[jira] [Comment Edited] (HDDS-5867) Update quota when deleting open keys
[ https://issues.apache.org/jira/browse/HDDS-5867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517954#comment-17517954 ]
mingchao zhao edited comment on HDDS-5867 at 4/6/22 8:52 AM:
-------------------------------------------------------------
bq. Yes, my point is that I do not think this pre-allocation approach is correct for two reasons:
bq. It is not possible (or at least difficult with some hacking required) to check that the open key's original intended bucket is actually the one being updated.
bq. Ozone is an object store with "last put wins" semantics. If two identical keys are being written, both writes happen simultaneously with no regard for each other and whoever puts last overwrites whoever puts first. It would make sense for quotas, which are calculated using key sizes, to follow the same semantics as the keys themselves with no stock being put in open key sizes.
bq. I think it would be both simpler and correct for quota updates to happen on key commit only.
Thansk [~erose] for your feedback. I get you point.
We add pre-allocation to check quota is to preventing IO waste. So far this seems to have bugs.
I agree with you. Since the Delete Key Service can delete the extra key, we can only update the usedByte when commit key. In this way, failed keys are not counted in the usedByte（we‘ll add service in the background to delete them), which is reasonable for the user.
was (Author: micahzhao):
bq. Yes, my point is that I do not think this pre-allocation approach is correct for two reasons:
It is not possible (or at least difficult with some hacking required) to check that the open key's original intended bucket is actually the one being updated.
Ozone is an object store with "last put wins" semantics. If two identical keys are being written, both writes happen simultaneously with no regard for each other and whoever puts last overwrites whoever puts first. It would make sense for quotas, which are calculated using key sizes, to follow the same semantics as the keys themselves with no stock being put in open key sizes.
I think it would be both simpler and correct for quota updates to happen on key commit only.
Thansk [~erose] for your feedback. I get you point.
We add pre-allocation to check quota is to preventing IO waste. So far this seems to have bugs.
I agree with you. Since the Delete Key Service can delete the extra key, we can only update the usedByte when commit key. In this way, failed keys are not counted in the usedByte（we‘ll add service in the background to delete them), which is reasonable for the user.
> Update quota when deleting open keys
> ------------------------------------
>
> Key: HDDS-5867
> URL: https://issues.apache.org/jira/browse/HDDS-5867
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: mingchao zhao
> Assignee: Kaijie Chen
> Priority: Major
> Labels: pull-request-available
>
> In HDDS-4120, we plan to delete expired data in Open key Tables. Request and Response are implemented in HDDS-4122.
> But we did not update UsedBytes and UsedNamespace in new request, we need to fix this problem.
> When these keys are deleted, the UsedBytes and UsedNamespace also need to be updated.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org