You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@systemml.apache.org by "Glenn Weidner (JIRA)" <ji...@apache.org> on 2017/09/09 01:20:00 UTC
[jira] [Updated] (SYSTEMML-1627) Mlogreg fails with file not found
on MNIST480m and certain mem configs
[ https://issues.apache.org/jira/browse/SYSTEMML-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Glenn Weidner updated SYSTEMML-1627:
------------------------------------
Fix Version/s: (was: SystemML 1.0)
SystemML 0.15
> Mlogreg fails with file not found on MNIST480m and certain mem configs
> ----------------------------------------------------------------------
>
> Key: SYSTEMML-1627
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1627
> Project: SystemML
> Issue Type: Bug
> Affects Versions: SystemML 0.14
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Fix For: SystemML 0.15
>
>
> Scenario: MultiLogReg over MNIST480m (480M rows x 784, sparse) fails for certain memory configurations (where unary operations over 480Mx2 intermediates run in CP and binary operations in SPARK), with the following exception:
> {code}
> Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 261 and 273 -- Error evaluating instruction: SPARK°tak+*°Y·MATRIX·DOUBLE°_mVar432·MATRIX·DOUBLE°1·SCALAR·INT·true°_Var437·SCALAR·DOUBLE
> at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:322)
> at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:221)
> at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:167)
> at org.apache.sysml.runtime.controlprogram.WhileProgramBlock.execute(WhileProgramBlock.java:165)
> ... 14 more
> Caused by: org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://node:8020/tmp/scratch_space/_p123456_1.2.34.56/_t0/temp154_56
> at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:287)
> at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)
> at org.apache.hadoop.mapred.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:45)
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> {code}
> The root cause is a missing export on guarded parallelize (as introduced in the 0.14 release) of cached matrices which have previously been collected from input rdds. These matrix objects are not marked dirty and hence not exported although they do not have an associated hdfs file yet.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)