You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/06/07 23:30:16 UTC

[I] Close Files.walk stream in GitVersionControlLocalFileStorage.deleteRepo [texera]

aglinxinyuan opened a new issue, #5548:
URL: https://github.com/apache/texera/issues/5548

   ## Background
   
   While reviewing #5447, Yicong-Huang flagged that the `Files.walk(...)` stream-leak pattern Copilot caught in my test cleanup helpers may exist elsewhere. A sweep of the codebase confirmed one production usage that leaks the underlying directory handle:
   
   `common/workflow-core/src/main/scala/org/apache/texera/amber/core/storage/util/dataset/GitVersionControlLocalFileStorage.java`, lines 80-85:
   
   ```java
   public static void deleteRepo(Path directoryPath) throws IOException {
       Files.walk(directoryPath)
           .sorted(Comparator.reverseOrder())
           .map(Path::toFile)
           .forEach(File::delete);
   }
   ```
   
   `Files.walk(...)` returns a closeable `java.util.stream.Stream` backed by an open directory handle. Without an explicit `close()`, the handle stays open until GC — which can flake temp-dir deletion on Windows and leak file descriptors on long-lived JVMs (e.g. the dataset service that calls `deleteRepo` whenever a Git-backed dataset is removed).
   
   Every other `Files.walk` / `Files.list` usage in the codebase already wraps the stream in `try/finally` and closes it (verified across `PveManager.scala`, `HuggingFaceModelResource.scala`, `HuggingFaceModelResourceSpec.scala`).
   
   ## What needs to change
   
   Convert `deleteRepo` to use try-with-resources so the stream is closed even if iteration throws:
   
   ```java
   public static void deleteRepo(Path directoryPath) throws IOException {
       try (var stream = Files.walk(directoryPath)) {
           stream
               .sorted(Comparator.reverseOrder())
               .map(Path::toFile)
               .forEach(File::delete);
       }
   }
   ```
   
   No behavior change for callers; just the stream-lifecycle fix.
   
   ## Scope
   
   - Single-method edit in `GitVersionControlLocalFileStorage.java`.
   - No new tests required (existing dataset deletion paths already exercise this code).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Close Files.walk stream in GitVersionControlLocalFileStorage.deleteRepo [texera]

Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang closed issue #5548: Close Files.walk stream in GitVersionControlLocalFileStorage.deleteRepo
URL: https://github.com/apache/texera/issues/5548


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Close Files.walk stream in GitVersionControlLocalFileStorage.deleteRepo [texera]

Posted by "lie18uci (via GitHub)" <gi...@apache.org>.
lie18uci commented on issue #5548:
URL: https://github.com/apache/texera/issues/5548#issuecomment-4686270957

   /take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Close Files.walk stream in GitVersionControlLocalFileStorage.deleteRepo [texera]

Posted by "justinsiek (via GitHub)" <gi...@apache.org>.
justinsiek commented on issue #5548:
URL: https://github.com/apache/texera/issues/5548#issuecomment-4654514487

   /take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org