[GitHub] [pinot] wirybeaver commented on pull request #10815: Periodically delete Tmp Segment file brought by the SplitCommit End phase

["wirybeaver (via GitHub)" <gi...@apache.org>]

wirybeaver commented on PR #10815:
URL: https://github.com/apache/pinot/pull/10815#issuecomment-1635074752

   > Some high level questions:
   > 
   > 1. Should we always rely on the async cleanup and remove the sync cleanup (assuming listing files is expensive)? I can see we can integrate more clean up logic into this periodic task in the future. Also, let's make the periodic task name more generic for future prove
   > 2. Do we create temp file for non-split-commit scenario?
   
   
   For question 1, I already ensured the sync cleanup will be skipped if async deletion is enabled.
   
   ```
       if (!isTmpSegmentAsyncDeletionEnabled()) 
         try {
           for (String uri : pinotFS.listFiles(tableDirURI, false)) {
             if (uri.contains(SegmentCompletionUtils.getSegmentNamePrefix(segmentName))) {
               LOGGER.warn("Deleting temporary segment file: {}", uri);
               Preconditions.checkState(pinotFS.delete(new URI(uri), true), "Failed to delete file: %s", uri);
   ```
   
   Rename it as TmpSegmentCleaner?
   
   For question 2, I need to inspect the source code.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org