[GitHub] [incubator-pinot] kkrugler commented on issue #7082: Fix handling of missing Deleted_Segments directory in SegmentDeletionManager

[GitBox <gi...@apache.org>]

kkrugler commented on issue #7082:
URL: https://github.com/apache/incubator-pinot/issues/7082#issuecomment-867175051


   Hi @mcvsubbu 
   
   > @kkrugler the call to remove aged deleted segments, is invoked from retention manager. If you never ever delete anything from any table in your cluster, you can (for now) disable RetentionManager. Alternatively, create a test table with a segment and delete it so that the parent dir is created on deletion. This will avoid your warnings and alerts.
   
   Yes, I've manually created the directory, so the warnings and alerts are gone.
   
   > It seems to be that HadoopPinotFS is miscoded. The method isDirectory() is a boolean, and if it does not exist or is not a directory, it should return false. IMO the fix is in the underlying FS implementation.
   
   Agreed, as that would be more consistent with how Java's `File.isDirectory()` works. I'll file a separate issue for that.
   
   Though even if this method worked as expected, and returned false, we'd get a warning in the logs every 5 minutes or so.
   
   Which is why I agree with @Jackie-Jiang that we should try to auto-create the directory, as then the warning is for a real problem (something went wrong with the creation, or it was accidentally deleted).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org