[GitHub] [incubator-druid] gianm commented on issue #7302: Druid Broker Result Level Cache Not Working Properly When Different Query Intervals Cover the Same Set of Segments

[GitBox <gi...@apache.org>]

gianm commented on issue #7302: Druid Broker Result Level Cache Not Working Properly When Different Query Intervals Cover the Same Set of Segments
URL: https://github.com/apache/incubator-druid/issues/7302#issuecomment-474909064
 
 
   I looked into this a bit and believe the ETag concept looks seriously flawed. It has one relatively minor bug - the one you mentioned, where it is calculated based on segment identifiers (which include the entire segment interval) and does not include a sub-interval within a segment. This causes the behavior you are seeing.
   
   But a potential larger bug is that it is leveraging `QueryToolChest.computeCacheKey`, which is designed for segment-level caching and, importantly, only includes query parameters that might affect the segment-level results. This is done to promote higher cache hit rates. For example, it _doesn't_ include things like:
   
   - grandTotal or postAggregations for timeseries
   - any non-sorting postAggregations for topN
   - having, subtotalsSpec, postAggregations, or limitSpec for groupBy (although the lack of limitSpec is a bug, I think, since groupBy supports limit push down in some cases now)
   
   I think this means we need a new method in the QueryToolChest for computing result-level cache keys.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org