[GitHub] [druid] kfaraz commented on issue #6329: CostBalancerStrategy.pickSegmentToMove can pick the same segment twice

[GitBox <gi...@apache.org>]

kfaraz commented on issue #6329:
URL: https://github.com/apache/druid/issues/6329#issuecomment-1208955556

   This is now fixed because the implementation takes the reservoir size and returns an iterator over those segments in a one-shot sampling.
   
   ```java
   default Iterator<BalancerSegmentHolder> pickSegmentsToMove(
         List<ServerHolder> serverHolders,
         Set<String> broadcastDatasources,
         int reservoirSize
     );
   ```
   
   ```java
   static List<BalancerSegmentHolder> getRandomBalancerSegmentHolders(
         final List<ServerHolder> serverHolders,
         Set<String> broadcastDatasources,
         int k
     )
     {
       List<BalancerSegmentHolder> holders = new ArrayList<>(k);
       int numSoFar = 0;
   
       for (ServerHolder server : serverHolders) {
         if (!server.getServer().getType().isSegmentReplicationTarget()) {
           // if the server only handles broadcast segments (which don't need to be rebalanced), we have nothing to do
           continue;
         }
   
         for (DataSegment segment : server.getServer().iterateAllSegments()) {
           if (broadcastDatasources.contains(segment.getDataSource())) {
             // we don't need to rebalance segments that were assigned via broadcast rules
             continue;
           }
   
           if (numSoFar < k) {
             holders.add(new BalancerSegmentHolder(server.getServer(), segment));
             numSoFar++;
             continue;
           }
           int randNum = ThreadLocalRandom.current().nextInt(numSoFar + 1);
           if (randNum < k) {
             holders.set(randNum, new BalancerSegmentHolder(server.getServer(), segment));
           }
           numSoFar++;
         }
       }
       return holders;
     }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org