[GitHub] [arrow] assignUser commented on issue #36161: [R] Problem retrieving memory used after gc() using arrow library

["assignUser (via GitHub)" <gi...@apache.org>]

assignUser commented on issue #36161:
URL: https://github.com/apache/arrow/issues/36161#issuecomment-1652704841

   This reads like an [XY problem](https://xyproblem.info/). To avoid miscommunication: @kevinpemonon what is your specific problem? Are you running an arrow script on a machine and running out of memory or something? If you suspect or have observed something like a memory leak a reprex would be great (e.g. a script that can get R oom killed where it would usually not be expected).
   
   Memory management, as well explained by Weston and Dewey is quite complex and often not as straight forward as one would think e.g. memory is kept allocated to a program like R by the os even if it is not needed at the moment to avoid costly operations (shuffling things around in ram to defrag) though this memory will be freed once it is requested by another program (but not if there are x GB free that can be used). And the environment the code is run on also plays a roll.
   
   All of this makes very general exploratory questions hard to answer.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org