You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "rich7420 (via GitHub)" <gi...@apache.org> on 2026/08/29 03:20:44 UTC

[PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

rich7420 opened a new pull request, #11151:
URL: https://github.com/apache/ozone/pull/11151

   ## What changes were proposed in this pull request?
   
   `ContainerUtils.getChunkDir` built a `new File(chunksPath)` and called `exists()` on the container's chunks directory on every WriteChunk and ReadChunk (reached via `ContainerLayoutVersion.getChunkFile`). The chunks directory is fixed for the container's lifetime, and on the read path this stat is pure added work.
   
   This resolves and validates the chunks directory once, caches the `File` on `ContainerData`, and reuses it on the data plane. `setChunksPath` invalidates the cache, so create / import / move still re-resolve. The cache field is `transient` and not in the YAML whitelist, so it is never serialized. It is `volatile` because `getChunkDir` is called concurrently from WriteChunk/ReadChunk threads (the memoization would be correct as a plain field too, since `File`'s fields are final, but `volatile` avoids repeated resolves under contention). The public getter/setter pair mirrors the existing `dbFile` accessor on `KeyValueContainerData`.
   
   Behavior is preserved: the first access still throws `UNABLE_TO_FIND_DATA_DIR` on a null path or a missing directory; a directory that disappears mid-life is still caught by the subsequent `open()`. The container scanner (`KeyValueContainerCheck`) stats the path independently and is unaffected.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-16326
   
   ## How was this patch tested?
   
   - Unit: `mvn -pl :hdds-container-service test -Dtest=TestContainerUtils,TestFilePerBlockStrategy,TestFilePerChunkStrategy` — `TestContainerUtils` 16 run, 0 failed; `TestFilePerBlockStrategy` 19, 0 failed; `TestFilePerChunkStrategy` 11, 0 failed
   - New in `TestContainerUtils`: `getChunkDirCachesResolvedDirectory` (caching + invalidation on `setChunksPath`) and `getChunkDirThrowsWhenChunksDirMissing` (`UNABLE_TO_FIND_DATA_DIR` on first resolve)
   - `checkstyle` and `pmd` clean
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-5490122539

   Thanks @chihsuan. The `getChunkDir` stat doubled as the write-path failure probe, so caching it for all callers dropped the UNHEALTHY-on-write path. Fixed: validate on writes, cache only on reads (`getChunksDirForRead`). `testUnhealthyContainer` passes, CI green.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-5461217322

   Quick JMH of a page-cache-hot `ReadChunk` (open + read) with the per-op `getChunkDir` stat (old) vs the cached dir (new), JDK 21 / macOS (5 forks x 5, n=25):
   
   ```
     chunk    old        new       saved
      4KB     11.2 us    9.35 us   1.85 us  (~16%)
      1MB     36.0 us    33.7 us   2.26 us  (~6%)
   ```
   
   The removed work is ~one stat plus a `File` allocation (~2 us, roughly constant), so it is a meaningful fraction for small cached reads and shrinks as the read grows. On the read path the stat is pure added work (the read otherwise goes straight to open()). No fsync in this run, and macOS stat is pricier than Linux, so the fraction is an upper bound. Standalone JMH, not part of the PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-5988749040

   @chungen0126 could you take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub) via issues" <is...@ozone.apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-6744605308

   Merged master, since the old base made fork CI fail in the Develocity Maven extension before building anything. I also kept the read-cache note in one place, explained the directory re-check before throwing, and dropped a duplicate test.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-6475478692

   On JDK 21 with compressed references, the estimated overhead is ~40 bytes per cached container (~38 MiB for 1M containers), reusing the existing path string. Agreed on handling short-circuit reads in a separate patch.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #11151:
URL: https://github.com/apache/ozone/pull/11151#issuecomment-5480935972

   @chihsuan thanks for the reminder 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-16326. Cache the container chunks directory instead of stat-ing it on every chunk operation [ozone]

Posted by "chihsuan (via GitHub)" <gi...@apache.org>.
chihsuan commented on code in PR #11151:
URL: https://github.com/apache/ozone/pull/11151#discussion_r3905358936


##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerData.java:
##########
@@ -263,6 +269,26 @@ public String getChunksPath() {
    */
   public void setChunksPath(String chunkPath) {
     this.chunksPath = chunkPath;
+    this.chunksDirFile = null;
+  }
+
+  /**
+   * Read-path accessor for the chunks directory. Resolves and validates it once
+   * via {@link ContainerUtils#getChunkDir(ContainerData)}, then returns the
+   * cached result on later calls to skip the per-read stat. Writes and other
+   * callers use {@code ContainerUtils.getChunkDir} directly, so a missing
+   * directory still surfaces as a storage failure on every operation.
+   *
+   * @return the resolved chunks directory
+   * @throws StorageContainerException if the chunks directory cannot be resolved
+   */
+  public File getChunksDirForRead() throws StorageContainerException {

Review Comment:
    Should this getter be `@JsonIgnore`?  



##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/FilePerChunkStrategy.java:
##########
@@ -224,8 +224,10 @@ public ChunkBufferToByteString readChunk(Container container, BlockID blockID,
     HddsVolume volume = containerData.getVolume();
 
     // In version1, we verify checksum if it is available and return data
-    // of the chunk file.
-    File finalChunkFile = getChunkFile(kvContainer, blockID, info);
+    // of the chunk file. Reads use the cached chunks directory to skip the
+    // per-read stat; writes keep validating so a missing directory is detected.
+    File finalChunkFile = FILE_PER_CHUNK.getChunkFile(
+        containerData.getChunksDirForRead(), blockID, info.getChunkName());

Review Comment:
   Should we still validate the directory at this path to preserve the previous error and logging behavior?



##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java:
##########
@@ -298,7 +298,10 @@ public static File getContainerFile(File containerBaseDir) {
   }
 
   /**
-   * Get the chunk directory from the containerData.
+   * Resolve and validate the chunk directory from the containerData. The
+   * directory's existence is checked on every call, so a chunk operation
+   * against a failed volume surfaces as a storage failure; write paths rely on
+   * this to mark the container unhealthy.

Review Comment:
   nit: Could this Javadoc stay about what the method does?  It now describes what callers rely on, and the same note appears in other files.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org