You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/21 03:56:29 UTC
[GitHub] [doris] dutyu opened a new issue, #11062: [Bug] exec_mem_limit option can not restrict the memory strictly
dutyu opened a new issue, #11062:
URL: https://github.com/apache/doris/issues/11062
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
Doris version:
- 1.1.0-rc5
Linux version:
- CentOS Linux release 7.4.1708 (Core)
Machine Specifications:
- Memory: 320GB
- CPU: 48 cores
Cluster Info:
- 7 Backends
- 3 Frontends (Use 16GB Heap)
### What's Wrong?
I hava executed a sql that use about 7 big tables which use left join (line number is all above 800 millions),
and set the option "**exec_mem_limit=16G**" and "**enable_vectorized_engine=false**".
But i found that the used memory of backends was up to **150GB** and then the sql failed, but the backends was still alive.
Error message is : `**ERROR 1105 (HY000): errCode = 2, detailMessage = Memory exceed limit. Cannot perform aggregation at node with id 1. Failed to allocate 312 bytes for intermediate tuple. Backend: 10.160.136.234, fragment: 408de0e268d448d6-b9eebf12171d1c5e Used: 2495175956, Limit: -1. You can change the limit by ses
MySQL [dp_data_db]> show backends;**`
And when i set the option "**exec_mem_limit=16G**" and "**enable_vectorized_engine=true**" and executed the same sql,
sql failed too, used memory of backends was up to **150GB**, and the backends was **killed by OOM Killer**.
### What You Expected?
Because we can not restrict the sqls that users to execute, so set a proper value for `**exec_mem_limit**` option is a very important method to avoid a serious cluster service problem. But now this method is not very useful, we will be very grateful if this problem is fixed as soon as possible.
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dutyu closed issue #11062: [Bug] Backend will be killed by oom killer cause exec_mem_limit option can not restrict the memory strictly
Posted by "dutyu (via GitHub)" <gi...@apache.org>.
dutyu closed issue #11062: [Bug] Backend will be killed by oom killer cause exec_mem_limit option can not restrict the memory strictly
URL: https://github.com/apache/doris/issues/11062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] dataalive commented on issue #11062: [Bug] exec_mem_limit option can not restrict the memory strictly
Posted by GitBox <gi...@apache.org>.
dataalive commented on issue #11062:
URL: https://github.com/apache/doris/issues/11062#issuecomment-1191536472
this problem will be fixed with mem-tracker in the following version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org