[jira] [Commented] (SPARK-7151) Correlation methods for DataFrame

["Reynold Xin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512917#comment-14512917 ] 

Reynold Xin commented on SPARK-7151:
------------------------------------

Seems like the first would make more sense in Scala/Java/Python, while the 2nd one would make more sense in R.

> Correlation methods for DataFrame
> ---------------------------------
>
>                 Key: SPARK-7151
>                 URL: https://issues.apache.org/jira/browse/SPARK-7151
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML, SQL
>            Reporter: Joseph K. Bradley
>            Priority: Minor
>              Labels: dataframe
>
> We should support computing correlations between columns in DataFrames with a simple API.
> This could be a DataFrame feature:
> {code}
> myDataFrame.corr("col1", "col2")
> // or
> myDataFrame.corr("col1", "col2", "pearson") // specify correlation type
> {code}
> Or it could be an MLlib feature:
> {code}
> Statistics.corr(myDataFrame("col1"), myDataFrame("col2"))
> // or
> Statistics.corr(myDataFrame, "col1", "col2")
> {code}
> (The first Statistics.corr option is more flexible, but it could cause trouble if a user tries to pass in 2 unzippable DataFrame columns.)
> Note: R follow the latter setup.  I'm OK with either.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org