[GitHub] [pinot] wirybeaver commented on pull request #10815: [Draft] Periodical Tmp Segment file deletion

["wirybeaver (via GitHub)" <gi...@apache.org>]

wirybeaver commented on PR #10815:
URL: https://github.com/apache/pinot/pull/10815#issuecomment-1612220804

   > Do we need to clean up tmp files periodically, or just when the controller starts? Will controller create extra tmp files during the regular life cycle without shut down?
   
   Hi Jackie, there are two kind of segment uploader during SegmentUpload phase for split commit.
   1. Server2ControllerSegmentUploader, controller upload the tmp file <seg_name>.tmp.<uuid> to deep store.
   2. PinotFSSegmentUploader, server upload the tmp file <seg_name><uuid> to deep store.
   
   In the CommitEnd phase, the deep storage can have outage due to flaky network when controller attempt to rename the tmp file. If peer downloading is enabled, controller will continue to commit the segment in ZK. The tmp file is left there in the regular lifecycle.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org