[jira] [Commented] (SPARK-6634) Allow replacing columns in Transformers

["Tree Field (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904445#comment-15904445 ] 

Tree Field commented on SPARK-6634:
-----------------------------------

I want this feature too.
because I often overwrite UnaryTransformer by myself  to enable this.

It seems it's only prevented in transformSchema method.
Now, unlike before v1.4,  dataframe's withColumn method used in UnaryTransformer allows replacing  the input column.

Any other reasons that is not allowed in transoformer, especially in UnaryTransformer.





> Allow replacing columns in Transformers
> ---------------------------------------
>
>                 Key: SPARK-6634
>                 URL: https://issues.apache.org/jira/browse/SPARK-6634
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 1.3.0
>            Reporter: Joseph K. Bradley
>            Priority: Minor
>
> Currently, Transformers do not allow input and output columns to share the same name.  (In fact, this is not allowed but also not even checked.)
> Short-term proposal: Disallow input and output columns with the same name, and add a check in transformSchema.
> Long-term proposal: Allow input & output columns with the same name, and where the behavior is that the output columns replace input columns with the same name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org