You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/12/01 10:31:15 UTC
[GitHub] [incubator-doris] Skysheepwang opened a new issue #4995: [Proposal] Separate Page Cache by page type
Skysheepwang opened a new issue #4995:
URL: https://github.com/apache/incubator-doris/issues/4995
Currently, all types of pages (including DATA_PAGE, INDEX_PAGE, DICT_PAGE, SHORT_KEY_PAGE) in segment_v2 are cached by StoragePageCache in one single memory space. In many query situations, we expect to reuse the index pages in cache, but which are probably replaced by the large amount of data pages due to the LRU algorithm. Even in some extreme cases, the high frequency of LRU replacement usage caused by data pages will bring great pressure to the system.
A separated page cache could be used to solve this problem.
Implementation:
- Change the "_cache" member of StoragePageCache class into "_data_cache" (for DATA) and "_index_page" (for INDEX, DICT, SHORT_KEY) and Add an "_index_cache_ratio" member as for the ratio of index cache.
- Add a configuration item "index_cache_ratio" to set the ratio. The range is: 0 <= ratio <= 1, 0/1 stands for only data/index pages are cached.
- Add "page_type" as an argument when reading pages, so that the StoragePageCache could use it to allocate the cache.
Advantages:
- By separating data and index pages, the hit rate of index pages could be highly improved and by tuning the ratio the hit rate of data pages will not be much decreased.
- By setting the config::index_cache_ratio (including 0 or 1), users can adapt the cache ratio to fit their query situations.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #4995: [Proposal] Separate Page Cache by page type
Posted by GitBox <gi...@apache.org>.
morningman closed issue #4995:
URL: https://github.com/apache/incubator-doris/issues/4995
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] xinghuayu007 commented on issue #4995: [Proposal] Separate Page Cache by page type
Posted by GitBox <gi...@apache.org>.
xinghuayu007 commented on issue #4995:
URL: https://github.com/apache/incubator-doris/issues/4995#issuecomment-736425626
it looks wonderful!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org