[GitHub] [ozone] captainzmc commented on pull request #3286: HDDS-6556. Update usedBytes only when commit key.

[GitBox <gi...@apache.org>]

captainzmc commented on PR #3286:
URL: https://github.com/apache/ozone/pull/3286#issuecomment-1091985151

   Thanks @sodonnel review and discuss this. My original idea is same with you, but there are some problems. 
   
   Ozone is only visible after the key has been written. If the key fails to write, eventually the key is unavailable. In addition, it is rare for a user to write more data than quota (while quota error happened, user needs to increase quota), also writing large keys is very unlikely to happen in practice.
   
   The main reasons for adopting the current way is:
   1. From the user's point of view, it is not reasonable to count a failed write key as quota. Because they can't do anything with the failed key. They don't know these failure keys exist. They cannot view, modify, or continue writing. These failed keys are like system garbage in Ozone.
   
   2. Each time we allocate, because we do not know the final size of the key, we increase usedBytes by 256*3MB each time. If the user's quota is only 600MB and the user only wants to write a 1MB key, the write will still fail. This doesn't make sense from the user's point of view because he has enough quota left to write 1MB. (In our production environment we have seen users raise this issue)
   
   3. At present, the only service we have implemented to clean up these failed keys is also background automatic. It's not up to the user to call it. So cleaning these keys is also the work of the Ozone system itself. If we add usedBytes in the allocate block, then there is no way for the user to manually release these usedBytes after the failure, and the user will be confused.
   
   4. Currently, when deleting a bucket, we require the user to clear all keys, but not open keys. At present, it seems that the background asynchronous clean Open key is the most efficient, exposing the interface for users to clean will make the use of Ozone more complicated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org