[GitHub] [incubator-druid] leventov opened a new issue #8125: Leave a single CachePopulator and choose sync/async mode automatically

[GitBox <gi...@apache.org>]

leventov opened a new issue #8125: Leave a single CachePopulator and choose sync/async mode automatically
URL: https://github.com/apache/incubator-druid/issues/8125
 
 
   In this issue, I list several ideas regarding refactoring/improvement of `CachePopulator` subsystem.
   
   1. It seems to me that `BackgroundCachePopulator` and `ForegroundCachePopulator` would be much more intuitively called Sync/Async ("background cache population" and "foreground cache population" terms in docs should then be updated to "synchronous/asynchronous cache population").
   
   2. There could be a single implementation. The executor may be `DirectExecutorService` in the "sync" case.
   
   3. It seems to me that the crucial difference between sync and async is whether blocking cache (Redis/Memcached) or local, (mostly) non-blocking cache (Caffeine) is used. At least, this should be highlighted in docs; at most, the mode of cachePopulator should be chosen automatically depending on the configured type of the cache.
   
   4. Does it make any sense to make async cache populator which pushes results to remote cache to have more than 1 thread? After all, we will be hitting the same bottleneck - the local NIC. So async populator may just have a queue and a single thread takes cache entries from the queue and tries to push them to remote cache.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org