You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@systemml.apache.org by "Matthias Boehm (JIRA)" <ji...@apache.org> on 2018/03/07 08:26:00 UTC

[jira] [Deleted] (SYSTEMML-2172) Repartitioning before caching ulta-sparse matrices

     [ https://issues.apache.org/jira/browse/SYSTEMML-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Boehm deleted SYSTEMML-2172:
-------------------------------------


> Repartitioning before caching ulta-sparse matrices
> --------------------------------------------------
>
>                 Key: SYSTEMML-2172
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2172
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: Matthias Boehm
>            Priority: Major
>
> Ultra-sparse matrices have dedicated serialized block representation which means that their in-memory storage in CSR can be much larger than on disk which leads to a blow-up of 128MB partitions to >1GB partitions. Accordingly, we should repartition the data before the initial caching in order to remove memory pressure and exploit the full parallelism.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)