[GitHub] [pinot] siddharthteotia commented on pull request #7898: Add help methods to check if segment needs reprocessing

[GitBox <gi...@apache.org>]

siddharthteotia commented on pull request #7898:
URL: https://github.com/apache/pinot/pull/7898#issuecomment-993108389


   I don't fully understand the optimization here. Looks like the new flow will be `ImmutableSegmentLoader` will call `SegmentPReprocessor.needsPreprocess()` and if this returns true, will the segment be pre-processed by the server during load. However, the code is using segment directory interface which seems to imply that segment exists on the server unless this is being done specifically for a custom/plugin implementation different from SegmentLocalFSDirectory ? 
   
   In that case I can see the point of just looking at the metadata (whether the column has index or not, whether column exists or not etc)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org