[GitHub] [druid] abhishekagarwal87 commented on issue #13006: It seams that cache does not help Improve query efficiency

[GitBox <gi...@apache.org>]

abhishekagarwal87 commented on issue #13006:
URL: https://github.com/apache/druid/issues/13006#issuecomment-1233845002

   There is no caching for scan queries. I didn't know it beforehand but as I think about it, it makes sense to not cache the results for scan queries. For the following reasons
   - Results for scan queries are streamed back to the client. Caching the results of the scan query is going to put memory pressure on the broker. 
   - There is no benefit of caching scan queries. For a group-by query, you could be potentially reading millions of rows and packing them into a few rows. The advantage of caching can be significant, especially at the segment level. (https://druid.apache.org/docs/0.23.0/querying/caching.html). With scan, it's unlikely that the user is going to scan millions of rows back to the client. Now one can say that we could potentially save the time spent on filtering. But that cost is cheap enough because of the bitmap indices that segments have. 
   
   what is the target number you are shooting for? are most of your queries going to scan queries or group-by queries? If the latter, you can re-run the same tests with group-by queries and see the difference. There is also some work done recently to improve QPS that is going to be available in the 24.0.0 release. One such change is - https://github.com/apache/druid/pull/12514


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org