[jira] [Created] (SYSTEMML-1569) Test MLContext for robustness and scalability

["Niketan Pansare (JIRA)" <ji...@apache.org>]

Niketan Pansare created SYSTEMML-1569:
-----------------------------------------

             Summary: Test MLContext for robustness and scalability
                 Key: SYSTEMML-1569
                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1569
             Project: SystemML
          Issue Type: Test
    Affects Versions: SystemML 1.0
            Reporter: Niketan Pansare


As more APIs are getting built on top of MLContext and with large-scale demos using MLContext and notebooks, we should test MLContext for robustness and scalability. The goal is that using MLContext should have constant overhead compared to commandline execution (both using similar formats).

As an example: we should check for potential OOM in Script History logic: https://github.com/apache/incubator-systemml/blob/master/src/main/java/org/apache/sysml/api/mlcontext/MLContextUtil.java#L902

If we uncomment https://github.com/apache/incubator-systemml/blob/master/src/main/java/org/apache/sysml/api/mlcontext/MLContextUtil.java#L897-L901, then you should get an OOM when passing large Numpy array with Python MLContext. This is because toString() method on MatrixBlock converts double [] into String.

[~deron] [~mwdusenb@us.ibm.com] [~reinwald] [~mboehm7]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)