[GitHub] [incubator-pinot] mcvsubbu commented on pull request #6250: Add server configuration to lazy load files from deep storage.

[GitBox <gi...@apache.org>]

mcvsubbu commented on pull request #6250:
URL: https://github.com/apache/incubator-pinot/pull/6250#issuecomment-725003691


   > Can we separate the unrelated changes to another PR?
   > 
   > IMO, the lazy load should be implemented as following:
   > 
   > * Load the metadata, but keep the index in deep store
   > * When segment gets queried, load index on demand
   > 
   > The current way of downloading all the segments, uncompress and load them at query time won't really work. @kishoreg wdyt?
   
   I think it is best to document these approaches in a design document instead of PR. This is a fairly involved feature, so I suggest writing up a document and soliciting comments before putting up the PR. Let me know if there was a document and I missed it.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org