You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "xinyiZzz (via GitHub)" <gi...@apache.org> on 2023/06/28 16:41:28 UTC
[GitHub] [doris] xinyiZzz opened a new pull request, #21311: [fix](memory) fix conf memory mode & fix GC more grace logging
xinyiZzz opened a new pull request, #21311:
URL: https://github.com/apache/doris/pull/21311
## Proposed changes
Issue Number: close #xxx
1. Fix conf memory mode,
- if performance, disable GC and Allocator memory check.
- if moderate, default, same as before.
- if compact, low/warning water mark is doubled, and there is no longer an interval between two GCs.
2. More understandable log after memory exceed limit
3. Optimize cache gc.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #21311:
URL: https://github.com/apache/doris/pull/21311#discussion_r1249984441
##########
be/src/vec/common/allocator.cpp:
##########
@@ -48,6 +49,9 @@ void Allocator<clear_memory_, mmap_populate, use_mmap>::sys_memory_check(size_t
size, doris::thread_context()->thread_mem_tracker()->label(),
doris::thread_context()->thread_mem_tracker_mgr->last_consumer_tracker(),
doris::MemTrackerLimiter::process_limit_exceeded_errmsg_str());
+ if (size > 1024l * 1024 * 1024) { // 1G
+ err_msg += "\nAlloc Stacktrace:\n" + doris::get_stack_trace();
Review Comment:
我加个条件，如果 size 大于系统剩余可用内存 && 已经不会再wait gc，则打日志，
目的是如果内存超限后瞬间OOM，找到那个大内存申请
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617687098
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617791257
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 40.82 seconds
stream load tsv: 454 seconds loaded 74807831229 Bytes, about 157 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 56 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 68.6 seconds inserted 10000000 Rows, about 145K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230703100708_clickbench_pr_171383.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1614454422
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617086377
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [improve](memory) more grace logging for memory exceed limit
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1619894784
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz merged pull request #21311: [improve](memory) more grace logging for memory exceed limit
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz merged PR #21311:
URL: https://github.com/apache/doris/pull/21311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [improve](memory) more grace logging for memory exceed limit
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1619753760
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix GC more grace logging
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1619742251
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21311: [improve](memory) more grace logging for memory exceed limit
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1619851843
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 39.45 seconds
stream load tsv: 457 seconds loaded 74807831229 Bytes, about 156 MB/s
stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select: 69.4 seconds inserted 10000000 Rows, about 144K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230704090744_clickbench_pr_171935.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617108427
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 39.72 seconds
stream load tsv: 464 seconds loaded 74807831229 Bytes, about 153 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 56 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select: 70.1 seconds inserted 10000000 Rows, about 142K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230703015758_clickbench_pr_171066.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617437195
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1614060290
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix conf memory mode & fix GC more grace logging
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1611766180
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix conf memory mode & fix GC more grace logging
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1611756901
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617688973
TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 39.62 seconds
stream load tsv: 454 seconds loaded 74807831229 Bytes, about 157 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 57 seconds loaded 1101869774 Bytes, about 18 MB/s
stream load parquet: 29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select: 68.0 seconds inserted 10000000 Rows, about 147K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230703091202_clickbench_pr_171297.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617095185
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617090044
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [improve](memory) more grace logging for memory exceed limit
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1619894697
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #21311:
URL: https://github.com/apache/doris/pull/21311#discussion_r1248356336
##########
be/src/vec/common/allocator.cpp:
##########
@@ -48,6 +49,9 @@ void Allocator<clear_memory_, mmap_populate, use_mmap>::sys_memory_check(size_t
size, doris::thread_context()->thread_mem_tracker()->label(),
doris::thread_context()->thread_mem_tracker_mgr->last_consumer_tracker(),
doris::MemTrackerLimiter::process_limit_exceeded_errmsg_str());
+ if (size > 1024l * 1024 * 1024) { // 1G
+ err_msg += "\nAlloc Stacktrace:\n" + doris::get_stack_trace();
Review Comment:
这么打log 可能会崩溃，1G 的估计内存分配挺多，然后get stack trace 可能卡住
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1614460757
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #21311:
URL: https://github.com/apache/doris/pull/21311#discussion_r1249984441
##########
be/src/vec/common/allocator.cpp:
##########
@@ -48,6 +49,9 @@ void Allocator<clear_memory_, mmap_populate, use_mmap>::sys_memory_check(size_t
size, doris::thread_context()->thread_mem_tracker()->label(),
doris::thread_context()->thread_mem_tracker_mgr->last_consumer_tracker(),
doris::MemTrackerLimiter::process_limit_exceeded_errmsg_str());
+ if (size > 1024l * 1024 * 1024) { // 1G
+ err_msg += "\nAlloc Stacktrace:\n" + doris::get_stack_trace();
Review Comment:
我加个条件，如果 size 大于系统剩余可用内存 && 外面没有catch bad alloc，则打日志，
目的是如果内存超限后瞬间OOM，找到那个大内存申请
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617545584
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21311:
URL: https://github.com/apache/doris/pull/21311#issuecomment-1617696641
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #21311: [fix](memory) fix GC more grace logging & fix cache GC
Posted by "xinyiZzz (via GitHub)" <gi...@apache.org>.
xinyiZzz commented on code in PR #21311:
URL: https://github.com/apache/doris/pull/21311#discussion_r1249984441
##########
be/src/vec/common/allocator.cpp:
##########
@@ -48,6 +49,9 @@ void Allocator<clear_memory_, mmap_populate, use_mmap>::sys_memory_check(size_t
size, doris::thread_context()->thread_mem_tracker()->label(),
doris::thread_context()->thread_mem_tracker_mgr->last_consumer_tracker(),
doris::MemTrackerLimiter::process_limit_exceeded_errmsg_str());
+ if (size > 1024l * 1024 * 1024) { // 1G
+ err_msg += "\nAlloc Stacktrace:\n" + doris::get_stack_trace();
Review Comment:
我加个条件，如果 size 大于1G && 外面没有catch bad alloc，则打日志，
目的是如果内存超限后瞬间OOM，找到那个大内存申请，大于1G的内存不catch bad alloc是不合理的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org