[jira] [Commented] (SYSTEMML-453) JMLC in Standalone mode and dependency on Hadoop

["Deron Eriksson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SYSTEMML-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048706#comment-16048706 ] 

Deron Eriksson commented on SYSTEMML-453:
-----------------------------------------

Looking over the code, it looks like Hadoop is very tightly coupled into certain data structures such as MatrixBlock and OutputInfo. As a result, it is probably hard to remove Hadoop classes completely when using JMLC, but it should be possible to greatly reduce the memory footprint required to run JMLC using an in-memory jar. The need for most Hadoop classes (such as those in org.apache.hadoop.conf) when using JMLC can probably be eliminated with some refactoring.

I'll have a look at it.

cc [~reinwald] [~mboehm7]


> JMLC in Standalone mode and dependency on Hadoop
> ------------------------------------------------
>
>                 Key: SYSTEMML-453
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-453
>             Project: SystemML
>          Issue Type: Improvement
>          Components: APIs
>            Reporter: Laura Chiticariu
>            Assignee: Deron Eriksson
>
> Hello,
> I am using the JMLC API in standalone mode. But Instantiating org.apache.sysml.api.jmlc.Connection fails with NoClassDefFoundError: org/apache/hadoop/conf/Configuration.
> Is it possible to remove the dependency on Hadoop when running SystemML in standalone mode?
> Thanks,
> Laura



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)