[GitHub] [pinot] jackjlli commented on pull request #8441: Add retainInstancesSequence feature to table rebalance to minimize data movement between instances

[GitBox <gi...@apache.org>]

jackjlli commented on pull request #8441:
URL: https://github.com/apache/pinot/pull/8441#issuecomment-1085383347


   > I don't follow why we need to store the pool to instances mapping in the ZK. Essentially what we need is to pass the existing instance partitions to the `InstanceReplicaGroupPartitionSelector.selectInstances()`, and add a flag to enable `minimizeMovement` and use a greedy algorithm to retain as much instances as possible from the candidate instances from each pool.
   
   Yes the initial plan is to reuse the existing instance partitions, but the thing is that for the instance which no longer exists in the current list of instance configs (which is gone), there is no way to know which pool it came from; i.e. even though we know there is an empty seat there, we don't know which pool we should select the new instance from.
   As long as we can control the input parameters of `replicaPartitionSelector.selectInstances()` (the input instancePartitions is always empty, thus as long as we control the instance sequence of `poolToInstanceConfigsMap`), we can always get the deterministic instancePartitions as output.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org