You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Felix Schüler (JIRA)" <ji...@apache.org> on 2016/12/19 20:18:58 UTC
[jira] [Created] (SYSTEMML-1158) MLContext values not found when
data is read in DML script
Felix Schüler created SYSTEMML-1158:
---------------------------------------
Summary: MLContext values not found when data is read in DML script
Key: SYSTEMML-1158
URL: https://issues.apache.org/jira/browse/SYSTEMML-1158
Project: SystemML
Issue Type: Bug
Components: APIs
Affects Versions: SystemML 0.11
Environment: IBM DSX
Reporter: Felix Schüler
When executing a DML script through MLContext that reads data from a file, resulting variables can not be found in the MLContext because they are removed from the context. This does not happen when the data is initialized randomly inside the script.
This example works fine with MLContext.getTuple
{code}
data = rand(100, 10)
c = ncol(data)
r = nrow(data)
stats = rand(rows=1, cols=c)
for (i in 0:(r - 1)) {
stats = (stats + data[i + 1,])
}
stats = (stats / r)
{code}
When I replace the random matrix initialization with {code} read("~/abalone_transformed", format="csv"){code} I get the following exception:
{code}
Name: org.apache.sysml.api.mlcontext.MLContextException
Message: Variable 'stats' not found
StackTrace: at org.apache.sysml.api.mlcontext.MLResults.getData(MLResults.java:103)
at org.apache.sysml.api.mlcontext.MLResults.outputValue(MLResults.java:1996)
at org.apache.sysml.api.mlcontext.MLResults.getTuple(MLResults.java:663)
{code}
The instructions for the working case:
{code}
PROGRAM ( size CP/SP = 28/0 )
--MAIN PROGRAM
----GENERIC (lines 1-4) [recompile=false]
------CP createvar _mVar167234 scratch_space//_p664_10.143.133.52//_t0/temp83636 true MATRIX binaryblock 3 3 1000 1000 9 copy
------CP rand 3 3 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 48 _mVar167234.MATRIX.DOUBLE
------CP createvar _mVar167235 scratch_space//_p664_10.143.133.52//_t0/temp83637 true MATRIX binaryblock 1 3 1000 1000 3 copy
------CP rand 1 3 1000 1000 0.0 1.0 1.0 -1 uniform 1.0 48 _mVar167235.MATRIX.DOUBLE
------CP assignvar 3.SCALAR.INT.true c.SCALAR.INT
------CP assignvar 3.SCALAR.INT.true r.SCALAR.INT
------CP cpvar _mVar167234 data
------CP cpvar _mVar167235 stats
------CP rmvar _mVar167234
------CP rmvar _mVar167235
----GENERIC (lines 5-7) [recompile=false]
----FOR (lines 5-7)
------GENERIC (lines 6-6) [recompile=false]
--------CP + i.SCALAR.INT.false 1.SCALAR.INT.true _Var167236.SCALAR.INT
--------CP createvar _mVar167237 scratch_space//_p664_10.143.133.52//_t0/temp83638 true MATRIX binaryblock 1 3 1000 1000 -1 copy
--------CP rangeReIndex data.MATRIX.DOUBLE _Var167236.SCALAR.INT.false _Var167236.SCALAR.INT.false 1.SCALAR.INT.true 3.SCALAR.INT.true _mVar167237.MATRIX.DOUBLE
--------CP rmvar _Var167236
--------CP createvar _mVar167238 scratch_space//_p664_10.143.133.52//_t0/temp83639 true MATRIX binaryblock 1 3 1000 1000 -1 copy
--------CP + stats.MATRIX.DOUBLE _mVar167237.MATRIX.DOUBLE _mVar167238.MATRIX.DOUBLE
--------CP rmvar _mVar167237
--------CP rmvar stats
--------CP cpvar _mVar167238 stats
--------CP rmvar _mVar167238
----GENERIC (lines 8-11) [recompile=false]
------CP createvar _mVar167239 scratch_space//_p664_10.143.133.52//_t0/temp83640 true MATRIX binaryblock 1 3 1000 1000 -1 copy
------CP / stats.MATRIX.DOUBLE 3.SCALAR.INT.true _mVar167239.MATRIX.DOUBLE
------CP rmvar stats
------CP cpvar _mVar167239 stats
------CP rmvar _mVar167239
------CP rmvar r
------CP rmvar c
------CP rmvar stats
{code}
Failing case (note that this is in a notebook reading a file from object storage)
{code}
PROGRAM ( size CP/SP = 38/7 )
--MAIN PROGRAM
----GENERIC (lines 1-4) [recompile=true]
------CP createvar pREADdata ~/abalone_transformed false MATRIX csv -1 -1 -1 -1 -1 copy false , true 0.0
------CP createvar _mVar150511 scratch_space//_p664_10.143.133.52//_t0/temp75273 true MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK csvrblk pREADdata.MATRIX.DOUBLE _mVar150511.MATRIX.DOUBLE 1000 1000 false , true 0.0
------CP createvar _mVar150512 scratch_space//_p664_10.143.133.52//_t0/temp75274 true MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK chkpoint _mVar150511.MATRIX.DOUBLE _mVar150512.MATRIX.DOUBLE MEMORY_AND_DISK
------CP rmvar _mVar150511
------CP cpvar _mVar150512 data
------CP rmvar _mVar150512
----GENERIC (lines 1-4) [recompile=true]
------CP ncol data.MATRIX.DOUBLE.false _Var150513.SCALAR.INT
------CP nrow data.MATRIX.DOUBLE.false _Var150514.SCALAR.INT
------CP createvar _mVar150515 scratch_space//_p664_10.143.133.52//_t0/temp75275 true MATRIX binaryblock 1 -1 1000 1000 -1 copy
------SPARK rand 1 ¶_Var150513¶ 1000 1000 0.0 1.0 1.0 -1 scratch_space/_p664_10.143.133.52//_t0/ uniform 1.0 _mVar150515.MATRIX.DOUBLE
------CP assignvar _Var150513.SCALAR.INT.false c.SCALAR.INT
------CP assignvar _Var150514.SCALAR.INT.false r.SCALAR.INT
------CP rmvar _Var150513
------CP rmvar _Var150514
------CP cpvar _mVar150515 stats
------CP rmvar _mVar150515
----GENERIC (lines 5-7) [recompile=true]
------CP createvar _mVar150516 scratch_space//_p664_10.143.133.52//_t0/temp75276 true MATRIX binaryblock -1 -1 1000 1000 -1 copy
------SPARK chkpoint data.MATRIX.DOUBLE _mVar150516.MATRIX.DOUBLE MEMORY_AND_DISK
------CP rmvar data
------CP cpvar _mVar150516 data
------CP rmvar _mVar150516
----FOR (lines 5-7)
------CP - r.SCALAR.INT.false 1.SCALAR.INT.true _Var150517.SCALAR.INT
------CP rmvar _Var150517
------GENERIC (lines 6-6) [recompile=true]
--------CP + i.SCALAR.INT.false 1.SCALAR.INT.true _Var150518.SCALAR.INT
--------CP ncol data.MATRIX.DOUBLE.false _Var150519.SCALAR.INT
--------CP createvar _mVar150520 scratch_space//_p664_10.143.133.52//_t0/temp75277 true MATRIX binaryblock 1 -1 1000 1000 -1 copy
--------SPARK rangeReIndex data.MATRIX.DOUBLE _Var150518.SCALAR.INT.false _Var150518.SCALAR.INT.false 1.SCALAR.INT.true _Var150519.SCALAR.INT.false _mVar150520.MATRIX.DOUBLE MULTI_BLOCK
--------CP rmvar _Var150518
--------CP rmvar _Var150519
--------CP createvar _mVar150521 scratch_space//_p664_10.143.133.52//_t0/temp75278 true MATRIX binaryblock 1 -1 1000 1000 -1 copy
--------SPARK + stats.MATRIX.DOUBLE _mVar150520.MATRIX.DOUBLE _mVar150521.MATRIX.DOUBLE
--------CP rmvar _mVar150520
--------CP rmvar stats
--------CP cpvar _mVar150521 stats
--------CP rmvar _mVar150521
----GENERIC (lines 8-11) [recompile=true]
------CP createvar _mVar150522 scratch_space//_p664_10.143.133.52//_t0/temp75279 true MATRIX binaryblock 1 -1 1000 1000 -1 copy
------SPARK / stats.MATRIX.DOUBLE r.SCALAR.INT.false _mVar150522.MATRIX.DOUBLE
------CP rmvar stats
------CP cpvar _mVar150522 stats
------CP rmvar _mVar150522
------CP rmvar r
------CP rmvar c
------CP rmvar stats
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)