[GitHub] [helix] subprotocol commented on issue #1888: Ephemeral Instance Support

[GitBox <gi...@apache.org>]

subprotocol commented on issue #1888:
URL: https://github.com/apache/helix/issues/1888#issuecomment-934605069


   > Can we understand why you need this? We can think about other solutions to support you requirement.
   
   Absolutely!  I would like to run a portion of my Helix participants on spot instances.  Spot instances allow one to cheaply scale processing when there is spare capacity.  The downside is that spot instances come and go, and when exactly is beyond my control.  Storage is disaggregated, so rebalancing helix resources is not expensive.
   
   I am not sure how best to configure helix for this model.  If host machines are fixed, when a host first starts up it can register itself as a helix instance and use that as the participant.
   
   
   When the hosts are not fixed (spot instances), as the spot instances come and go, I would end up with a forever increasing set of helix instances in zookeeper.
   
   I understand and share your concern about high traffic on the zookeeper path.  Would a better strategy be to try and re-use helix instances that are offline when a spot instance first starts up?  Or is there a better strategy that I'm not seeing?
   
   Thank you @junkaixue!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org