You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/09/05 03:06:11 UTC
[GitHub] [doris] xinyiZzz opened a new pull request, #12332: [fix](memtracker) Fix the exceeded limit of the first query execution
xinyiZzz opened a new pull request, #12332:
URL: https://github.com/apache/doris/pull/12332
# Proposed changes
Issue Number: close #xxx
## Problem summary
In some cases, when the user executes the query for the first time, an error of the exceeded mem limit will be reported, and the query will be successful again.
This is because when the query is executed for the first time, the memory consumed by adding the page cache and other caches is recorded in the query mem tracker, and it is hoped that the behavior of multiple queries will be counted.
A temporary solution, remove the hook of scanner thread, test clickbench q13
1. Before removing the scanner thread hook
Enable page cache: 3G for the first query, 3G for the tracker; 900M for the second query, 900M for the tracker.
Turn off page cache: 1.9G for the first query, 1.9G for the tracker; 900M for the second query, 900M for the tracker
2. After removing the scanner thread hook
Enable page cache: 3G for the first query, 1.6G for the tracker; 900M for the second query, 900M for the tracker
Turn off page cache: 1.9G for the first query, 1.6G for the tracker; 900M for the second query, 900M for the tracker
TODO, a better solution is to track storage-related memory separately, in the scanner thread.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #12332: [fix](memtracker) Fix the exceeded limit of the first query execution
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on PR #12332:
URL: https://github.com/apache/doris/pull/12332#issuecomment-1236587183
> After remove the hook in olap scanner, the memtracker‘s value has different between first query and second query. There maybe some bugs.
fix bug in `MemTrackerLimiter::cache_consume_local`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12332: [fix](memtracker) Fix the exceeded limit of the first query execution
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12332:
URL: https://github.com/apache/doris/pull/12332#issuecomment-1236505696
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #12332: [fix](memtracker) Fix the exceeded limit of the first query execution
Posted by GitBox <gi...@apache.org>.
yiguolei merged PR #12332:
URL: https://github.com/apache/doris/pull/12332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12332: [fix](memtracker) Fix the exceeded limit of the first query execution
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12332:
URL: https://github.com/apache/doris/pull/12332#issuecomment-1236505717
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org