You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 12:36:12 UTC

[GitHub] [beam] damccorm opened a new issue, #19645: WindmillStateCache grossly misunderestimates object size in its weighting function

damccorm opened a new issue, #19645:
URL: https://github.com/apache/beam/issues/19645

   As a little experiment, we set the WindmillStateCache size to 2 GB.  After we let a job run for a few hours, the workers all died from OOMs.  Digging into the memory usage, over 10 GB of heap was used by the WindmillStateCache.  The total weight the cache had calculated however was only ~800 MB.
   
   It looks like the StateCacheEntry calculation is way off.  The weight of an entry was calculated as only ~200, while the actual heap usage of it (as reported by YourKit at least) was around 2,400 bytes.
   
   Imported from Jira [BEAM-7684](https://issues.apache.org/jira/browse/BEAM-7684). Original Jira may contain additional context.
   Reported by: SteveNiemitz.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org