You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/02/20 23:00:03 UTC
[GitHub] [druid] jihoonson opened a new issue #9383: Performance regression
in OvershadowableManager
jihoonson opened a new issue #9383: Performance regression in OvershadowableManager
URL: https://github.com/apache/druid/issues/9383
### Affected Version
Since 0.16.
### Description
`OvershadowableManager` was added in #7491 to manage all segments in the same time chunk. `addChunk()` is pretty slow when you have 1000+ segments per time chunk. Look at the below flame graph.
There are two slow parts in `addChunk()`.
- [It currently determines the state of the new atomicUpdateGroup by calling `overshadows()` against all existing segments](https://github.com/apache/druid/blob/master/core/src/main/java/org/apache/druid/timeline/partition/OvershadowableManager.java#L678-L682). This stream contributes 77% to the total overhead. This should be fixed by navigating overlapping visible groups directly rather than iterating all visible groups.
- `findLowestOverlappingEntry()` contributes the remaining 23%. This is to skip non-overlapping entries in the map. This iteration can be reduced by using more accurate indexes: `stateMap` uses `RootPartitionRange` as its key, but this can be split into two indexes of `rangeStart` and `rangeEnd`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson closed issue #9383: Performance regression in
OvershadowableManager
Posted by GitBox <gi...@apache.org>.
jihoonson closed issue #9383: Performance regression in OvershadowableManager
URL: https://github.com/apache/druid/issues/9383
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org