You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@systemml.apache.org by "Matthias Boehm (JIRA)" <ji...@apache.org> on 2018/02/24 09:38:00 UTC

[jira] [Closed] (SYSTEMML-2155) Recompute nnz on spark checkpoints for large matrices

     [ https://issues.apache.org/jira/browse/SYSTEMML-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Boehm closed SYSTEMML-2155.
------------------------------------
       Resolution: Fixed
         Assignee: Matthias Boehm
    Fix Version/s: SystemML 1.1

> Recompute nnz on spark checkpoints for large matrices
> -----------------------------------------------------
>
>                 Key: SYSTEMML-2155
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2155
>             Project: SystemML
>          Issue Type: Task
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>            Priority: Major
>             Fix For: SystemML 1.1
>
>
> Operations over large matrices with dimensions > max integer, are always forced to spark which implies missing nnz information due to lazy evaluation in spark. This task aims to proactively recompute the nnz for such large matrices (which are often ultra-sparse) to improve size propagation during recompilation and repeated access.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)