[GitHub] [pinot] siddharthteotia commented on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

[GitBox <gi...@apache.org>]

siddharthteotia commented on PR #8441:
URL: https://github.com/apache/pinot/pull/8441#issuecomment-1088338173

   > I wouldn't mix the problem this PR is trying to solve with the fault domain support. Essentially what we want this PR to achieve is to minimize the server shift when picking servers from the candidate servers. 
   
   +1. The intention was to indeed keep them separate and only solve the specific problem in this PR. The assumption of an entire replica group fitting into a pool will be violated in FD aware instance assignment but let's deal with it as part of the latter implementation (in-progress).
   
   Similarly, let's also not worry about adding / removing pool in this PR and add that as a TODO ?
   
   Lastly, regarding making this stateful v/s stateless -- I had the same question earlier in this PR https://github.com/apache/pinot/pull/8441#discussion_r839050322 primarily because of Znode size but in offline discussion with @jackjlli  I got convinced otherwise that it is not possible and wasn't aware pool selection is deterministic.
   
   Also I had suggested @jackjlli to refactor / wrap the new code into a separate tag pool selector as otherwise it was getting too convoluted and we anticipate more changes to same code as FD awareness comes in (which will also be put behind a new strategy / selector). It's done and now if we make it stateless, code will get simplified but may be we should still keep the separate selector ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org