[GitHub] [druid] himanshug commented on issue #8846: VersionedIntervalTimeline performance corner case with high number of overlapping segments

[GitBox <gi...@apache.org>]

himanshug commented on issue #8846: VersionedIntervalTimeline performance corner case with high number of overlapping segments
URL: https://github.com/apache/druid/issues/8846#issuecomment-591725587
 
 
   @jihoonson thanks for triggering the discussion :)
   
   I faced this issue because of the very specific way  segment intervals overlapped and that all historicals  were restarted simultaneously. This issue disappears if historicals are restarted one by one. That is what I opted as a work around to solve this problem because bringing down all historicals in prod simultaneously wasn't going to give customers a good experience anyway. Hence, this issue lost its importance somewhat. Did you face this in another context ?
   
   also, there is a caveat (on broker) and hence the approach needed discussion, operations in `VersionedIntervalTimeline` need to hold a  lock to  protect all data structures  .. currently lock is not held for long as we do things  one segment at a time .. this gives incoming queries a  chance to lookup segments and execute . With batching,  lock  could potentially be held for relatively longer and queries might block.
   
   Since the issue doesn't appear in regular druid deployment/upgrade use case but in situations where users is doing something major on druid cluster requiring bringing down all historicals, they could always just restart the broker/coordinator in the end as well. For this reason, I kept my prototype in prototype stage and didn't create a PR.
   
   That said, if this issue is surfacing in other use cases giving pain then it might be worth doing the batching. I could probably dig out the code and do a PR. However, at this point, I don't think it is necessary. 
   
   PS: I kept this issue open just in case someone else noticed this issue in another context.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org