You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/08/10 17:25:23 UTC
[GitHub] [doris] bin41215 opened a new pull request, #11658: [minor] turn virtual memory to physical memory
bin41215 opened a new pull request, #11658:
URL: https://github.com/apache/doris/pull/11658
# Proposed changes
In the check_sys_mem_info method of mem_tracker_limiter.h, turn virtual memory to physical memory
## Problem summary
The log information about memory is confusing
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [x] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [x] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [x] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [x] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [x] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] bin41215 closed pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
bin41215 closed pull request #11658: [minor](be)turn virtual memory to physical memory
URL: https://github.com/apache/doris/pull/11658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] yiguolei commented on a diff in pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
yiguolei commented on code in PR #11658:
URL: https://github.com/apache/doris/pull/11658#discussion_r943062723
##########
be/src/runtime/memory/mem_tracker_limiter.h:
##########
@@ -72,7 +72,7 @@ class MemTrackerLimiter final : public MemTracker {
if (PerfCounters::get_vm_rss() + bytes >= MemInfo::mem_limit()) {
return Status::MemoryLimitExceeded(
"{}: TryConsume failed, bytes={} process whole consumption={} mem limit={}",
- _label, bytes, MemInfo::current_mem(), MemInfo::mem_limit());
Review Comment:
Hi, currently memtracker only count virtual memory. And sometimes virtual memory is 3 times larger than physical memory.
I think we should include both virtual memory and physical memory in this error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] bin41215 commented on a diff in pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
bin41215 commented on code in PR #11658:
URL: https://github.com/apache/doris/pull/11658#discussion_r943380941
##########
be/src/runtime/memory/mem_tracker_limiter.h:
##########
@@ -72,7 +72,7 @@ class MemTrackerLimiter final : public MemTracker {
if (PerfCounters::get_vm_rss() + bytes >= MemInfo::mem_limit()) {
return Status::MemoryLimitExceeded(
"{}: TryConsume failed, bytes={} process whole consumption={} mem limit={}",
- _label, bytes, MemInfo::current_mem(), MemInfo::mem_limit());
Review Comment:
Thank you for your prompt reply.
What I understand is to print physical memory and virtual memory separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on PR #11658:
URL: https://github.com/apache/doris/pull/11658#issuecomment-1212619081
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] bin41215 commented on pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
bin41215 commented on PR #11658:
URL: https://github.com/apache/doris/pull/11658#issuecomment-1213115912
> LGTM, run `sh build-support/clang-format.sh`, and resubmit @bin41215
@xinyiZzz Sorry, my repo project was deleted by mistake, so I reapplied a pr again.
Please review this : https://github.com/apache/doris/pull/11747
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] bin41215 closed pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
bin41215 closed pull request #11658: [minor](be)turn virtual memory to physical memory
URL: https://github.com/apache/doris/pull/11658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on a diff in pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on code in PR #11658:
URL: https://github.com/apache/doris/pull/11658#discussion_r944034747
##########
be/src/runtime/memory/mem_tracker_limiter.h:
##########
@@ -72,7 +72,7 @@ class MemTrackerLimiter final : public MemTracker {
if (PerfCounters::get_vm_rss() + bytes >= MemInfo::mem_limit()) {
return Status::MemoryLimitExceeded(
"{}: TryConsume failed, bytes={} process whole consumption={} mem limit={}",
- _label, bytes, MemInfo::current_mem(), MemInfo::mem_limit());
Review Comment:
It was a bug to only print virtual memory before, because the use of physical memory to judge exceed is here.
Now be process virtual memory can be much larger than physical memory, I am investigating this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz commented on pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
xinyiZzz commented on PR #11658:
URL: https://github.com/apache/doris/pull/11658#issuecomment-1213370190
> > LGTM, run `sh build-support/clang-format.sh`, and resubmit @bin41215
>
> @xinyiZzz Sorry, my repo project was deleted by mistake, so I reapplied a pr again. Please review this : #11747
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xinyiZzz closed pull request #11658: [minor](be)turn virtual memory to physical memory
Posted by GitBox <gi...@apache.org>.
xinyiZzz closed pull request #11658: [minor](be)turn virtual memory to physical memory
URL: https://github.com/apache/doris/pull/11658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org