ML: embed a transformer

[Sa...@wellsfargo.com]

Hi all, I'm very new to spark and looking forward to get deep into the topic.

Right now I am trying to inherit my own transformer, by what I am reading so far, it is not very public that we can apply to this practice as "users".
I am defining my transformer based on the Binarizer, but simply failing to retrieve the private package ml.param.shared._ to use common traits. I've read it should be public since 1.4.x but not sure.

Thanks!
Saif

RE: ML: embed a transformer

[Sa...@wellsfargo.com]

Thank you, I will do as you suggested.
Ps: I read that in this random user archive I found: http://mail-archives.us.apache.org/mod_mbox/spark-user/201506.mbox/%3C55709F7B.2090906@gmail.com%3E

Saif

From: Feynman Liang [mailto:fliang@databricks.com]
Sent: Monday, September 14, 2015 4:08 PM
To: Ellafi, Saif A.
Cc: dev
Subject: Re: ML: embed a transformer

Where did you read that it should be public? The traits in ml.param.shared are meant to be used across internal spark.ml<http://spark.ml> transformer implementations.

If your transformer could be included in spark.ml<http://spark.ml>, then I would recommend implementing it there so these package private traits can be reused. Otherwise, you can re-use Param and Transformer which are both public.

On Mon, Sep 14, 2015 at 11:48 AM, <Sa...@wellsfargo.com>> wrote:
Hi all, I’m very new to spark and looking forward to get deep into the topic.

Right now I am trying to inherit my own transformer, by what I am reading so far, it is not very public that we can apply to this practice as “users”.
I am defining my transformer based on the Binarizer, but simply failing to retrieve the private package ml.param.shared._ to use common traits. I’ve read it should be public since 1.4.x but not sure.

Thanks!
Saif

Re: ML: embed a transformer

[Feynman Liang <fl...@databricks.com>]

Where did you read that it should be public? The traits in ml.param.shared
are meant to be used across internal spark.ml transformer implementations.

If your transformer could be included in spark.ml, then I would recommend
implementing it there so these package private traits can be reused.
Otherwise, you can re-use Param and Transformer which are both public.

On Mon, Sep 14, 2015 at 11:48 AM, <Sa...@wellsfargo.com> wrote:

> Hi all, I’m very new to spark and looking forward to get deep into the
> topic.
>
> Right now I am trying to inherit my own transformer, by what I am reading
> so far, it is not very public that we can apply to this practice as “users”.
> I am defining my transformer based on the Binarizer, but simply failing to
> retrieve the private package ml.param.shared._ to use common traits. I’ve
> read it should be public since 1.4.x but not sure.
>
> Thanks!
> Saif
>
>