You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Mike Dusenberry (JIRA)" <ji...@apache.org> on 2016/10/01 03:08:20 UTC

[jira] [Commented] (SYSTEMML-995) MLContext dataframe-frame conversion with index column & vector column

    [ https://issues.apache.org/jira/browse/SYSTEMML-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537756#comment-15537756 ] 

Mike Dusenberry commented on SYSTEMML-995:
------------------------------------------

Additionally, I'm running into null pointer exceptions regarding cache blocks during Spark mapmm operations that I believe could be related here the above mentioned block output info.  Unfortunately I'll have to wait for this job to run again in order to grab the stacktrace.

> MLContext dataframe-frame conversion with index column & vector column
> ----------------------------------------------------------------------
>
>                 Key: SYSTEMML-995
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-995
>             Project: SystemML
>          Issue Type: Bug
>          Components: APIs
>    Affects Versions: SystemML 0.11
>            Reporter: Matthias Boehm
>            Priority: Blocker
>
> MLContext currently always assumes data frame to frame conversion without existing index column. Since the user cannot communicate the existence of this column, the data conversion leads to incorrect results as an additional column is included in the output frame. We need make the MLContext handling of frames consistent with the handling of matrices.
> Additionally, the conversion code in {{MLContextConversionUtil.dataFrameToFrameObject()}} does not yet take into account frames with vectors, although the recent addition adds this support in the underlying {{FrameRDDConverterUtils.java}} class.  Therefore, the number of columns set when {{mc == null}} is incorrect.
> Thanks [~mwdusenb@us.ibm.com] for catching this issue. cc [~acs_s] [~deron]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)