You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "freemandealer (via GitHub)" <gi...@apache.org> on 2024/09/25 08:17:03 UTC
[I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
freemandealer opened a new issue, #41280:
URL: https://github.com/apache/doris/issues/41280
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
Occasionally, we found that there have been cases of disk cache data escaping from the management of doris file cache, causing disk space leaks. To make it easier for debugging, we need a checking tool that compares the contents in the doris file cache memory management structure with the current disk contents to identify the differences between the two (which are potential problematic data).
To better understand how file cache works, please refer to: https://doris.apache.org/zh-CN/docs/dev/compute-storage-decoupled/file-cache/ and https://www.bilibili.com/video/BV1ath9eGEqL
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-2440902583
> I'd like to work on it, please assign it to me
sure, thanks for you participation and welcome to Doris community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #41280: [Feature] Doris FileCache memory info vs disk info consistency check
URL: https://github.com/apache/doris/issues/41280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-3095103979
https://github.com/Lupinus/doris/commit/84a990d31626571d7dcb2646d2ce9243de9391c7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "Lupinus (via GitHub)" <gi...@apache.org>.
Lupinus commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-2440880218
I'd like to work on it, please assign it to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "Lupinus (via GitHub)" <gi...@apache.org>.
Lupinus commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-2451254373
Sorry for not expressing myself clearly, "Issue" refers to naming a function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "Lupinus (via GitHub)" <gi...@apache.org>.
Lupinus commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-2449893542
The main idea is to scan the disk once and then scan the _files again.
In FileCacheStorage, add a virtual function unordered_set checkConsistency(BlockFileCache* _mgr, lambda handler); where handler is a lambda used to handle inconsistent AccessKeyAndOffset entries, recording any inconsistencies found. An inconsistency means that an AccessKeyAndOffset exists only in either BlockFileCache or FSFileCacheStorage.
In the implementation of checkConsistency in FSFileCacheStorage, the main task is to iterate through the fileBlock directory items under _cache_base_path, checking for their existence in _files of BlockFileCache and whether their sizes are consistent. If an entry does not exist, the handler is called; if it exists, it is recorded in an unordered_set (used for the return value).
In BlockFileCache, add a function checkConsistency, which has two main parts. The first part calls the _storage’s checkConsistency, obtaining its return value (an unordered_set that records which AccessKeyAndOffset entries have already been found during the disk scan). The second part iterates through _files, and if any item is not found in the unordered_set, it calls the handler to record this inconsistency, ultimately returning these inconsistent items.
In terms of the API, in FileCacheAction, add two types of operations. One is to input a path and check the consistency of that path, which essentially calls BlockFileCache's checkConsistency. The second is to obtain all paths (to facilitate the use of the first operation).
Any suggestions regarding function naming? It is a frustrating issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "freemandealer (via GitHub)" <gi...@apache.org>.
freemandealer commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-2451215861
hi @Lupinus
> In FileCacheStorage, add a virtual function unordered_set checkConsistency(BlockFileCache* _mgr, lambda handler)
We don't need _mgr as its parameter since the call chain is as follows:
FileCacheAction -> BlockFileCache(holding _mutex) -> specific storage
> checking for their existence in _files of BlockFileCache and whether their sizes are consistent.
Additionally, please consider the consistency of their metadata. Please take a look at `FileBlock::cache_type()` & `FileBlock::expiration_time()`. These metadata are encoded into the directory name and file name in the filesystem and refer to `FSFileCacheStorage::load_cache_info_into_memory` for details.
BTW, when using unorderd_map, the map key should include all the above-mentioned info because the file path itself could be duplicated (but with different cache types or expiration_time).
> Is it ok for the API's return to be inconsistent file names and offsets?
Given that the inconsistency could be in two categories, i.e. missing in _files v.s. missing in filesystem, we should point that out along with the file path (not file name alone, should be file path) in the HTTP reponse.
> Any suggestions regarding function naming? It is a frustrating issue.
No problem with the naming. And what do you mean by 'a frustrating issue'? Is it too easy or hard for you? If there is any problem with the issue itself, please help me improve it. I appreciate your help in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] [Feature] Doris FileCache memory info vs disk info consistency check [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #41280:
URL: https://github.com/apache/doris/issues/41280#issuecomment-3764516689
We're closing this issue because it hasn't been updated in a while.
If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org