[jira] [Commented] (SPARK-13068) Extend pyspark ml paramtype conversion to support lists

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178794#comment-15178794 ] 

Joseph K. Bradley commented on SPARK-13068:
-------------------------------------------

I like the idea of automatic casting when possible.  A few comments:
* I don't think we should add special subclasses of Param for each type.  We had to do that in Scala to make it Java-friendly.  Instead, the casting could be handled by a single method which takes the value and the expected type.
* I don't want to go overboard on conversions.  E.g., if a {{str}} is required, I don't think we should try casting to a string.  (If a user calls {{setInputCol([1,2,3])}}, then they are probably making a mistake.)
* I don't think we should add ParamValidators in Python at this time.  We could consider doing it later, but it's really a separate issue.  I'm also worried about having validation in 2 places since they could get out of synch.

How does that sound?

> Extend pyspark ml paramtype conversion to support lists
> -------------------------------------------------------
>
>                 Key: SPARK-13068
>                 URL: https://issues.apache.org/jira/browse/SPARK-13068
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML, PySpark
>            Reporter: holdenk
>            Priority: Trivial
>
> In SPARK-7675 we added type conversion for PySpark ML params. We should follow up and support param type conversion for lists and nested structures as required. This blocks having all PySpark ML params having type information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org