You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "hanahmily (via GitHub)" <gi...@apache.org> on 2024/11/24 14:23:05 UTC

[I] [BanyanDB-Server] Add Memory Usage Control for BanyanDB Queries [skywalking]

hanahmily opened a new issue, #12798:
URL: https://github.com/apache/skywalking/issues/12798

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   BanyanDB queries can use significant memory, particularly when processing large datasets. This proposal introduces a mechanism to control the maximum memory consumption during query execution. By utilizing the statistical information available for each block, the system can summarize the total memory usage of ongoing queries and enforce limits to prevent out-of-memory (OOM) errors.
   
   A configurable parameter, such as `max_query_memory`, should be introduced to set the maximum allowable memory usage. If the estimated memory consumption exceeds this limit, the query execution should be gracefully terminated or paused.
   
   It is important to provide detailed error messages when a query is terminated due to memory constraints. Additionally, memory usage metrics should be made available, and memory usage details should be logged for debugging and optimization purposes.
   
   Optionally, a backpressure mechanism could be implemented to slow down or queue queries when memory usage approaches the set limit.
   
   Please note that series index queries should not be affected by this limitation.
   
   ### Use case
   
   _No response_
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a pull request to implement this on your own?
   
   - [ ] Yes I am willing to submit a pull request on my own!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [BanyanDB-Server] Add Memory Usage Control for BanyanDB Queries [skywalking]

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #12798: [BanyanDB-Server] Add Memory Usage Control for BanyanDB Queries
URL: https://github.com/apache/skywalking/issues/12798


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org