You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Matthias Boehm (JIRA)" <ji...@apache.org> on 2017/03/23 20:42:41 UTC
[jira] [Closed] (SYSTEMML-1430) Robust memory handling broadcasts
[ https://issues.apache.org/jira/browse/SYSTEMML-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm closed SYSTEMML-1430.
------------------------------------
> Robust memory handling broadcasts
> ---------------------------------
>
> Key: SYSTEMML-1430
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1430
> Project: SystemML
> Issue Type: Sub-task
> Components: APIs, Runtime
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Fix For: SystemML 1.0
>
>
> This task aims to make the memory handling of re-used and temporary broadcast variables more robust in order to avoid unnecessary OOMs.
> 1) Explicitly destroy temporary broadcast variables because we would otherwise never clean them up and Spark's ContextCleaner seems to be a best effort daemon that triggers every 30min or on garbage collection (which might be too late if a large object is allocated)
> 2) Keep track of currently softly reachable (re-used) broadcasts in order to take their size into account when deciding on guarded collect.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)