You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "chihsuan (via GitHub)" <gi...@apache.org> on 2026/08/23 04:19:30 UTC

[PR] HDDS-16175. Do not charge usedNamespace on hsync re-commit [ozone]

chihsuan opened a new pull request, #11092:
URL: https://github.com/apache/ozone/pull/11092

   ## What changes were proposed in this pull request?
   
   `OMKeyCommitRequest` increments `usedNamespace` after the whole if/else chain, so every commit charges a namespace unit, including a re-commit of a key the same client already hsync'd:
   
   https://github.com/apache/ozone/blob/8a81f47f66129283cd6124c178fcf16f48539722/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java#L324-L375
   
   `c351de9914` (HDDS-13756) hoisted the increment out of the `else` branch, which reverts the namespace half of `df4df20371` (HDDS-7965). The client commits once per newly allocated block plus once on close, while delete refunds one unit per key, so a file spanning N blocks leaves N units behind and the counter never returns to 0.
   
   This change moves `incrUsedNamespace(1L)` back inside the branches that add a key name, in `OMKeyCommitRequest` and `OMKeyCommitRequestWithFSO`. The overwrite branch keeps the charge because it has just refunded the previous key's units. The stale comment above the chain, which predates HDDS-6709, is replaced in both classes.
   
   Already drifted counters are not repaired by this patch and need `ozone repair om quota start`. During a rolling upgrade, mixed-version OMs apply different accounting to new commits, the same exposure HDDS-13756 introduced when it changed this counter.
   
   Bucket versioning is out of scope, per the discussion on HDDS-16127.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-16175
   
   ## How was this patch tested?
   
   Added coverage for:
   
   - The same key committed three times by one client, two hsync re-commits and the close, leaving `usedNamespace` at 1. Fails with 3 before the patch. `TestOMKeyCommitRequestWithFSO` inherits it, so both the OBS and FSO commit paths are covered.
   - One file written through ofs with an hsync per block, then closed and deleted. Fails with 5 before the patch. `TestHSync` uses an FSO bucket, so cluster-level coverage is FSO only.
   
   The neighbouring `testCommitWithHsyncIncrementalUsages` asserts `getUsedBytes()` only, which is why this went unnoticed.
   
   Ran:
   
   ```shell
   mvn -pl :ozone-manager test \
     -Dtest='TestOMKeyCommitRequest,TestOMKeyCommitRequestWithFSO' \
     -DskipShade -DskipRecon -DskipDocs
   
   mvn -pl :ozone-integration-test test \
     -Dtest=TestHSync#testUsedNamespaceWithRepeatedHsync \
     -DskipShade -DskipRecon -DskipDocs
   
   mvn -pl :ozone-manager,:ozone-integration-test checkstyle:check \
     -DskipShade -DskipRecon -DskipDocs
   ```
   
   Also ran `author.sh` successfully.
   
   Generated-by: Claude Code (Opus 5)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16175. Do not change usedNamespace on hsync re-commit [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #11092:
URL: https://github.com/apache/ozone/pull/11092#issuecomment-5408682282

   Thanks @chihsuan for the patch, @chungen0126, @rich7420 for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16175. Do not change usedNamespace on hsync re-commit [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #11092:
URL: https://github.com/apache/ozone/pull/11092


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org