[GitHub] [spark] HyukjinKwon commented on a diff in pull request #36205: [SPARK-38907][PYTHON] Implement DataFrame.corrwith

[GitBox <gi...@apache.org>]

HyukjinKwon commented on code in PR #36205:
URL: https://github.com/apache/spark/pull/36205#discussion_r851059280


##########
python/pyspark/pandas/frame.py:
##########
@@ -1310,6 +1310,138 @@ def corr(self, method: str = "pearson") -> "DataFrame":
         """
         return cast(DataFrame, ps.from_pandas(corr(self, method)))
 
+    def corrwith(
+        self, other: DataFrameOrSeries, drop: bool = True, method: str = "pearson"
+    ) -> "Series":
+        """
+        Compute pairwise correlation.
+
+        Pairwise correlation is computed between rows or columns of
+        DataFrame with rows or columns of Series or DataFrame. DataFrames
+        are first aligned along both axes before computing the
+        correlations.
+
+        .. versionadded:: 3.4.0
+
+        Parameters
+        ----------
+        other : DataFrame, Series
+            Object with which to compute correlations.
+
+        drop : bool, default False
+            Drop missing indices from result.
+
+        method : str, default 'pearson'
+            Method of correlation, one of:
+
+            * pearson : standard correlation coefficient
+
+        Returns
+        -------
+        Series
+            Pairwise correlations.
+
+        See Also
+        --------
+        DataFrame.corr : Compute pairwise correlation of columns.
+
+        Examples
+        --------
+        >>> from pyspark.pandas.config import set_option, reset_option
+        >>> set_option("compute.ops_on_diff_frames", True)

Review Comment:
   Maybe we can use `pyspark.pandas.config.option_context`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org