[GitHub] [incubator-pinot] chenboat commented on issue #3849: Using PinotFS interface instead of local file to download a segment f…

[GitBox <gi...@apache.org>]

chenboat commented on issue #3849: Using PinotFS interface instead of local file to download a segment f…
URL: https://github.com/apache/incubator-pinot/pull/3849#issuecomment-479703907
 
 
   > Also, the servers should pull from the URI that is there in the metadata. The way we should solve this is to change the URI in the metadata to point to a remote store for completed realtime segments
   
   Are you suggesting in case of non-local storage, Pinot servers by-pass Pinot controllers to download the segment from remote directly. I think that is a good idea (and we in fact talked out this during last meeting) but it is out of scope of this PR in my opinion because:
   (1) PinotSegmentUploadRestletResource right now does not work with Pinot FS -- so it is a bug we should fix unless we want to delete this resource.
   (2) Currently in LLC, in the common path the commit servers anyway need to pass segments to Pinot controller during commit. The download segment path is in fact a less common code path. So I do not see the point of doing the less common path optimization before we solve the whole problem of by-passing Pinot controller for segment data transfer.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org