[GitHub] [pinot] mcvsubbu commented on issue #7378: Support to add multiple directory location for this property in server (pinot.server.instance.dataDir)

[GitBox <gi...@apache.org>]

mcvsubbu commented on issue #7378:
URL: https://github.com/apache/pinot/issues/7378#issuecomment-915421214


   So, what you really need is not just multiple directory support, but (also)  support for some policy using which some directory may be chosen to keep segments in the server. Perhaps you are also looking for automated move of segments from "hot" to "warm" and then back to "hot" storage and also some policies there as well (you don't want to be moving segments back and forth all the time, since you also need cpu to serve queries).
   
   Perhaps Issue #7229  helps here? There is already a design doc in progress.
   
   Further, it may be better to support tiered storage, as per [this](https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit#heading=h.sbtdm9byxqnc) design, so that you can have "hot" and "warm servers that are appropriately priced. It will reduce the complicated code patterns introduced within the server by supporting multiple directories for local storage.
   
   Overall, -1 for this feature.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org