You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@systemml.apache.org by "Matthias Boehm (JIRA)" <ji...@apache.org> on 2016/09/30 20:41:20 UTC

[jira] [Resolved] (SYSTEMML-996) Unnecessary shuffle / block mismatch in frame right indexing

     [ https://issues.apache.org/jira/browse/SYSTEMML-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Boehm resolved SYSTEMML-996.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: SystemML 0.11

> Unnecessary shuffle / block mismatch in frame right indexing
> ------------------------------------------------------------
>
>                 Key: SYSTEMML-996
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-996
>             Project: SystemML
>          Issue Type: Bug
>    Affects Versions: SystemML 0.11
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>             Fix For: SystemML 0.11
>
>
> The implementation of our spark frame right indexing instruction unnecessarily tries to create aligned partial blocks, which causes unnecessary shuffle and merge issues (on merging these partial blocks) as frame can have variable block sizes. We should avoid merge/shuffle altogether and simply slice out the relevant sub blocks. 
> cc [~mwdusenb@us.ibm.com] [~acs_s]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)