You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Arvind Surve <ac...@yahoo.com.INVALID> on 2017/09/20 16:50:22 UTC
[DISCUSS] R-Interface to SystemML
Hi,
R4ML is an open source project which provides a R interface to SystemML.Its a bridge between SystemML and Spark-R.
Lets discuss here if and how we can get two code streams work together to benefit development/community effort.
Arvind Surve | Spark Technology Center | http://www.spark.tc/
Re: [DISCUSS] R-Interface to SystemML
Posted by Deron Eriksson <de...@gmail.com>.
I agree with Niketan. An R interface definitely makes sense for SystemML.
DML itself is based on R, so it's surprising we have Java/Scala/Python
interfaces to SystemML but we don't have an R interface.
Perhaps R4ML committers could supply a little more info? For instance:
1) Would they like to merge R4ML code into the main SystemML project
itself? (Currently we have no modules.)
2) What would they like to merge?
3) If so, how do they propose to do so?
4) Who will do the majority of the work to add R4ML code to SystemML? Or
who would like to volunteer to do this?
5) Who will maintain the contributed code? Or who would like to volunteer
to do this?
6) Documentation is needed (fit in SystemML documentation framework).
7) Testing is needed (fit into SystemML testing framework).
8) How is this packaged?
From a technology standpoint, I think an R interface totally makes sense.
As for a minor criticism (which I apply to other parts of SystemML too), I
see script wrappers at https://github.com/SparkTC/r4ml/tree/master/R4ML/R.
This tightly binds the existing DML scripts to R, which means DML
input/output modifications could potentially require modifications to R
code.
Deron
On Thu, Sep 21, 2017 at 11:00 AM, Niketan Pansare <np...@us.ibm.com>
wrote:
> Janardhan: I believe this is the R4ML repo: https://github.com/SparkTC/
> r4ml . Arvind: please correct me if I am wrong.
>
> Overall, having a R interface for SystemML is an awesome idea. Since I am
> not an R4ML expert, may be R4ML committers can comment on how they envision
> "two code streams to work together".
>
> Also, comparing the features of R4ML with that of our Python APIs will be
> useful as it might make a stronger case for R4ML.
>
> As an FYI, here are different ways Python users can use SystemML:
> - Using MLContext to invoke DML script (http://apache.github.io/
> systemml/beginners-guide-python#invoking-dmlpydml-scripts-using-mlcontext
> and http://apache.github.io/systemml/spark-mlcontext-
> programming-guide.html)
> - Python algorithms wrappers (http://apache.github.io/
> systemml/beginners-guide-python#invoke-systemmls-algorithms)
> - (not important for R4ML discussion): Python DSL (
> http://apache.github.io/systemml/beginners-guide-python#matrix-operations)
>
> Thanks,
>
> Niketan Pansare
> IBM Almaden Research Center
> E-mail: npansar At us.ibm.com
> http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
>
> [image: Inactive hide details for Janardhan ---09/21/2017 04:44:02 AM---Hi
> Arvind, This is a great idea. One question: the R4ML generat]Janardhan
> ---09/21/2017 04:44:02 AM---Hi Arvind, This is a great idea. One question:
> the R4ML generates any plan like the SystemML with `D
>
> From: Janardhan <j1...@protonmail.com>
> To: Arvind Surve <ac...@yahoo.com.INVALID>, "dev@systemml.apache.org" <
> dev@systemml.apache.org>
> Date: 09/21/2017 04:44 AM
> Subject: Re: [DISCUSS] R-Interface to SystemML
> ------------------------------
>
>
>
> Hi Arvind,
>
> This is a great idea. One question: the R4ML generates any plan like the
> SystemML with `DML` Or with providing some interface we leverage this
> feature ?. Community effort in the sense of collaborative algorithm
> implementation.(?)
>
> Is this the Spark-R repo ( https://urldefense.proofpoint.
> com/v2/url?u=https-3A__github.com_rstudio_sparklyr&d=DwIGaQ&
> c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_
> 6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=6VRs_
> J7zXj9jZEavEP8iNvVfISAjDJeM8wFL2sBnb0g&e= ) ?
>
> Thanks,
> Janardhan
>
> Sent with [ProtonMail](https://urldefense.proofpoint.com/v2/
> url?u=https-3A__protonmail.com&d=DwIGaQ&c=jf_iaSHvJObTbx-
> siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=
> uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=khkGV3oXz1W5m_
> ueQRuKWlNMVOXXCVhV_ytNCINjJWY&e= ) Secure Email.
>
> > -------- Original Message --------
> > Subject: [DISCUSS] R-Interface to SystemML
> > Local Time: September 20, 2017 12:50 PM
> > UTC Time: September 20, 2017 4:50 PM
> > From: acs_s@yahoo.com.INVALID
> > To: dev@systemml.apache.org <de...@systemml.apache.org>
> >
> > Hi,
> > R4ML is an open source project which provides a R interface to
> SystemML.Its a bridge between SystemML and Spark-R.
> > Lets discuss here if and how we can get two code streams work together
> to benefit development/community effort.
> >
> > Arvind Surve | Spark Technology Center | https://urldefense.proofpoint.
> com/v2/url?u=http-3A__www.spark.tc_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=
> HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-
> PSwyvQzq5gwX1GL0&s=Yj8qfo7sjGBxX547UMGfLHRZFUxCAjZDTHWe8B7jLxI&e=
>
>
>
Re: [DISCUSS] R-Interface to SystemML
Posted by Niketan Pansare <np...@us.ibm.com>.
Janardhan: I believe this is the R4ML repo: https://github.com/SparkTC/r4ml
. Arvind: please correct me if I am wrong.
Overall, having a R interface for SystemML is an awesome idea. Since I am
not an R4ML expert, may be R4ML committers can comment on how they envision
"two code streams to work together".
Also, comparing the features of R4ML with that of our Python APIs will be
useful as it might make a stronger case for R4ML.
As an FYI, here are different ways Python users can use SystemML:
- Using MLContext to invoke DML script (
http://apache.github.io/systemml/beginners-guide-python#invoking-dmlpydml-scripts-using-mlcontext
and
http://apache.github.io/systemml/spark-mlcontext-programming-guide.html)
- Python algorithms wrappers (
http://apache.github.io/systemml/beginners-guide-python#invoke-systemmls-algorithms
)
- (not important for R4ML discussion): Python DSL (
http://apache.github.io/systemml/beginners-guide-python#matrix-operations)
Thanks,
Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar
From: Janardhan <j1...@protonmail.com>
To: Arvind Surve <ac...@yahoo.com.INVALID>,
"dev@systemml.apache.org" <de...@systemml.apache.org>
Date: 09/21/2017 04:44 AM
Subject: Re: [DISCUSS] R-Interface to SystemML
Hi Arvind,
This is a great idea. One question: the R4ML generates any plan like the
SystemML with `DML` Or with providing some interface we leverage this
feature ?. Community effort in the sense of collaborative algorithm
implementation.(?)
Is this the Spark-R repo (
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_rstudio_sparklyr&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=6VRs_J7zXj9jZEavEP8iNvVfISAjDJeM8wFL2sBnb0g&e=
) ?
Thanks,
Janardhan
Sent with [ProtonMail](
https://urldefense.proofpoint.com/v2/url?u=https-3A__protonmail.com&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=khkGV3oXz1W5m_ueQRuKWlNMVOXXCVhV_ytNCINjJWY&e=
) Secure Email.
> -------- Original Message --------
> Subject: [DISCUSS] R-Interface to SystemML
> Local Time: September 20, 2017 12:50 PM
> UTC Time: September 20, 2017 4:50 PM
> From: acs_s@yahoo.com.INVALID
> To: dev@systemml.apache.org <de...@systemml.apache.org>
>
> Hi,
> R4ML is an open source project which provides a R interface to
SystemML.Its a bridge between SystemML and Spark-R.
> Lets discuss here if and how we can get two code streams work together to
benefit development/community effort.
>
> Arvind Surve | Spark Technology Center |
https://urldefense.proofpoint.com/v2/url?u=http-3A__www.spark.tc_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=Yj8qfo7sjGBxX547UMGfLHRZFUxCAjZDTHWe8B7jLxI&e=
Re: [DISCUSS] R-Interface to SystemML
Posted by Janardhan <j1...@protonmail.com>.
Hi Arvind,
This is a great idea. One question: the R4ML generates any plan like the SystemML with `DML` Or with providing some interface we leverage this feature ?. Community effort in the sense of collaborative algorithm implementation.(?)
Is this the Spark-R repo ( https://github.com/rstudio/sparklyr ) ?
Thanks,
Janardhan
Sent with [ProtonMail](https://protonmail.com) Secure Email.
> -------- Original Message --------
> Subject: [DISCUSS] R-Interface to SystemML
> Local Time: September 20, 2017 12:50 PM
> UTC Time: September 20, 2017 4:50 PM
> From: acs_s@yahoo.com.INVALID
> To: dev@systemml.apache.org <de...@systemml.apache.org>
>
> Hi,
> R4ML is an open source project which provides a R interface to SystemML.Its a bridge between SystemML and Spark-R.
> Lets discuss here if and how we can get two code streams work together to benefit development/community effort.
>
> Arvind Surve | Spark Technology Center | http://www.spark.tc/