[jira] [Commented] (SPARK-12751) Traits generated by SharedParamsCodeGen should not be private

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093176#comment-15093176 ] 

Joseph K. Bradley commented on SPARK-12751:
-------------------------------------------

It'd be nice to get your feedback on the other JIRA.  I'm working on writing a design doc for this & hope to post it soon.

> Traits generated by SharedParamsCodeGen should not be private
> -------------------------------------------------------------
>
>                 Key: SPARK-12751
>                 URL: https://issues.apache.org/jira/browse/SPARK-12751
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 1.5.2, 1.6.0
>            Reporter: Wojciech Jurczyk
>            Priority: Minor
>
> Many Estimators and Transformers mix in traits generated by [SharedParamsCodeGen|https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala]. These estimators and transformers (like StringIndexer, MinMaxScaler etc) are accessible publicly while traits generated by SharedParamsCodeGen are private\[ml\]. From user's code it is possible to invoke methods that the traits introduce but it is illegal to use any trait explicitly. For example, you can call setInputCol(str) on StringIndexer but you are not allowed to assign StringIndexer to a variable of type HasInputCol.
> {code:java}
> val x: HasInputCol = new StringIndexer() // Usage of HasInputCol is illegal.
> {code}
> For example, it is impossible to create a collection of transformers that have both HasInputCol and HasOutputCol (e.g. Set\[Transformer with HasInputCol with HasOutputCol\]). We have to use structural typing and reflective calls like this:
> {code}
> ml.Estimator[_] { val outputCol: ml.param.Param[String] }
> {code}
> This seems easy to fix, exposing a couple of traits should not break anything. On the other hand, maybe it goes deeper than that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org