You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Hossein <fa...@gmail.com> on 2018/05/22 22:23:10 UTC

ML Pipelines in R

Hi all,

SparkR supports calling MLlib functionality with an R-friendly API. Since
Spark 1.5 the (new) SparkML API which is based on pipelines and parameters
has matured significantly. It allows users build and maintain complicated
machine learning pipelines. A lot of this functionality is difficult to
expose using the simple formula-based API in SparkR.

I just submitted a SPIP <https://issues.apache.org/jira/browse/SPARK-21190>
to propose a new R package, SparkML, to be distributed along with SparkR as
part of Apache Spark. Please view the JIRA ticket and provide feedback &
comments.

Thanks,
--Hossein

Re: ML Pipelines in R

Posted by Hossein <fa...@gmail.com>.

Correction: the SPIP is https://issues.apache.org/jira/browse/SPARK-24359


--Hossein

On Tue, May 22, 2018 at 6:23 PM, Hossein <fa...@gmail.com> wrote:

> Hi all,
>
> SparkR supports calling MLlib functionality with an R-friendly API. Since
> Spark 1.5 the (new) SparkML API which is based on pipelines and parameters
> has matured significantly. It allows users build and maintain complicated
> machine learning pipelines. A lot of this functionality is difficult to
> expose using the simple formula-based API in SparkR.
>
> I just submitted a SPIP
> <https://issues.apache.org/jira/browse/SPARK-21190> to propose a new R
> package, SparkML, to be distributed along with SparkR as part of Apache
> Spark. Please view the JIRA ticket and provide feedback & comments.
>
> Thanks,
> --Hossein
>