You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Matthias Boehm (JIRA)" <ji...@apache.org> on 2016/09/30 07:49:20 UTC
[jira] [Updated] (SYSTEMML-996) Unnecessary shuffle / block
mismatch in frame right indexing
[ https://issues.apache.org/jira/browse/SYSTEMML-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm updated SYSTEMML-996:
------------------------------------
Description:
The implementation of our spark frame right indexing instruction unnecessarily tries to create aligned partial blocks, which causes unnecessary shuffle and merge issues (on merging these partial blocks) as frame can have variable block sizes. We should avoid merge/shuffle altogether and simply slice out the relevant sub blocks.
cc [~mwdusenb@us.ibm.com] [~acs_s]
was:cc [~mwdusenb@us.ibm.com] [~acs_s]
> Unnecessary shuffle / block mismatch in frame right indexing
> ------------------------------------------------------------
>
> Key: SYSTEMML-996
> URL: https://issues.apache.org/jira/browse/SYSTEMML-996
> Project: SystemML
> Issue Type: Bug
> Affects Versions: SystemML 0.11
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
>
> The implementation of our spark frame right indexing instruction unnecessarily tries to create aligned partial blocks, which causes unnecessary shuffle and merge issues (on merging these partial blocks) as frame can have variable block sizes. We should avoid merge/shuffle altogether and simply slice out the relevant sub blocks.
> cc [~mwdusenb@us.ibm.com] [~acs_s]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)