[GitHub] [ozone] errose28 commented on a diff in pull request #3286: HDDS-6556. Check quota while allocate and update usedBytes when commit.

[GitBox <gi...@apache.org>]

errose28 commented on code in PR #3286:
URL: https://github.com/apache/ozone/pull/3286#discussion_r866283268


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java:
##########
@@ -196,36 +196,41 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
         }
       }
 
+      // If bucket versioning is turned on during the update, between key
+      // creation and key commit, old versions will be just overwritten and
+      // not kept. Bucket versioning will be effective from the first key
+      // creation after the knob turned on.
+      RepeatedOmKeyInfo oldKeyVersionsToDelete = null;
+      OmKeyInfo keyToDelete =
+          omMetadataManager.getKeyTable(getBucketLayout()).get(dbOzoneKey);
+      if (keyToDelete != null && !omBucketInfo.getIsVersionEnabled()) {
+        oldKeyVersionsToDelete = getOldVersionsToCleanUp(dbOzoneKey,
+            keyToDelete, omMetadataManager,
+            trxnLogIndex, ozoneManager.isRatisEnabled());
+      }
+
       omKeyInfo =
           omMetadataManager.getOpenKeyTable(getBucketLayout()).get(dbOpenKey);
       if (omKeyInfo == null) {
         throw new OMException("Failed to commit key, as " + dbOpenKey +
             "entry is not found in the OpenKey table", KEY_NOT_FOUND);
       }
+      omBucketInfo = getBucketInfo(omMetadataManager, volumeName, bucketName);
       omKeyInfo.setDataSize(commitKeyArgs.getDataSize());
-
       omKeyInfo.setModificationTime(commitKeyArgs.getModificationTime());
-
       // Update the block length for each block
-      List<OmKeyLocationInfo> allocatedLocationInfoList =
-          omKeyInfo.getLatestVersionLocations().getLocationList();
       omKeyInfo.updateLocationInfoList(locationInfoList, false);
-
       // Set the UpdateID to current transactionLogIndex
       omKeyInfo.setUpdateID(trxnLogIndex, ozoneManager.isRatisEnabled());
 
-      // If bucket versioning is turned on during the update, between key
-      // creation and key commit, old versions will be just overwritten and
-      // not kept. Bucket versioning will be effective from the first key
-      // creation after the knob turned on.
-      RepeatedOmKeyInfo oldKeyVersionsToDelete = null;
-      OmKeyInfo keyToDelete =
-          omMetadataManager.getKeyTable(getBucketLayout()).get(dbOzoneKey);
-      if (keyToDelete != null && !omBucketInfo.getIsVersionEnabled()) {
-        oldKeyVersionsToDelete = getOldVersionsToCleanUp(dbOzoneKey,
-            keyToDelete, omMetadataManager,
-            trxnLogIndex, ozoneManager.isRatisEnabled());
+      long correctedSpace = omKeyInfo.getReplicatedSize();
+      // Subtract the size of blocks to be overwritten.
+      if (keyToDelete != null) {
+        correctedSpace -= keyToDelete.getReplicatedSize();

Review Comment:
   @captainzmc I don't totally follow the logic in this part, and the corresponding part in OMKeyCommitRequestWithFSO. Could you clarify how this is handling quota calculations when we are/are not overwriting a key, and how versioning be enabled or disabled affects this?



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMAllocateBlockRequestWithFSO.java:
##########
@@ -144,7 +144,12 @@ public OMClientResponse validateAndUpdateCache(OzoneManager ozoneManager,
       long preAllocatedSpace = newLocationList.size()
               * ozoneManager.getScmBlockSize()
               * openKeyInfo.getReplicationConfig().getRequiredNodes();
-      checkBucketQuotaInBytes(omBucketInfo, preAllocatedSpace);
+      long hadAllocatedSpace =
+          openKeyInfo.getLatestVersionLocations().getLocationList().size()
+              * ozoneManager.getScmBlockSize()
+              * openKeyInfo.getReplicationConfig().getRequiredNodes();

Review Comment:
   I don't think this will give the correct result for EC. Looks like `QuotaUtil#getReplicatedSize` has logic that might help here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org