[jira] [Closed] (SPARK-7151) Correlation methods for DataFrame

["Reynold Xin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin closed SPARK-7151.
------------------------------
          Resolution: Duplicate
       Fix Version/s: 1.4.0
            Assignee: Burak Yavuz
    Target Version/s: 1.4.0

> Correlation methods for DataFrame
> ---------------------------------
>
>                 Key: SPARK-7151
>                 URL: https://issues.apache.org/jira/browse/SPARK-7151
>             Project: Spark
>          Issue Type: Sub-task
>          Components: ML, SQL
>            Reporter: Joseph K. Bradley
>            Assignee: Burak Yavuz
>            Priority: Minor
>              Labels: dataframe
>             Fix For: 1.4.0
>
>
> We should support computing correlations between columns in DataFrames with a simple API.
> This could be a DataFrame feature:
> {code}
> myDataFrame.corr("col1", "col2")
> // or
> myDataFrame.corr("col1", "col2", "pearson") // specify correlation type
> {code}
> Or it could be an MLlib feature:
> {code}
> Statistics.corr(myDataFrame("col1"), myDataFrame("col2"))
> // or
> Statistics.corr(myDataFrame, "col1", "col2")
> {code}
> (The first Statistics.corr option is more flexible, but it could cause trouble if a user tries to pass in 2 unzippable DataFrame columns.)
> Note: R follow the latter setup.  I'm OK with either.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org