You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/08/28 17:33:55 UTC

[GitHub] [druid] AmatyaAvadhanula commented on issue #10940: Coordinator loads all the data only in few historicals of a tier, leaving the rest almost empty

AmatyaAvadhanula commented on issue #10940:
URL: https://github.com/apache/druid/issues/10940#issuecomment-1229514991

   @mounikanakkala @liubo-it 
   I think the reason this occurs is because you had a datasource with a higher granularity (maybe monthly etc) and a new datasource with a lower granularity (hourly) was later being loaded.
    
   When new historicals were added, the cost for these was much lower than that for the historicals with higher granularity segments because of the nature of the cost function. This is why the older historicals were ignored and data was being assigned to the new ones as part of the loading phase.
    
   This is mitigated to a certain extent because of the following extra normalization in cachingCost:
   `    return cost * (server.getMaxSize() / server.getAvailableSize());
   `
    
   I think the cluster should eventually balance itself in both cases but there may be a temporary state of uneven distribution


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org