[GitHub] [arrow-datafusion] crepererum commented on issue #3941: Generate runtime errors if the memory budget is exceeded [EPIC]

[GitBox <gi...@apache.org>]

crepererum commented on issue #3941:
URL: https://github.com/apache/arrow-datafusion/issues/3941#issuecomment-1290334270

   I think it's OK to have some slack room on top of the limit, which is somewhat controlled by the batch size. It's unlikely that we are going to account for every single byte anyways, since there might be some aux data structures here and there that are heap-allocated. So I would treat this is a "best effort limit w/o impacting performance (too much)".
   
   We could later (if there's demand for it) add a config option "strict memory enforcement" that impacts performance.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org