[GitHub] [spark] itholic commented on a change in pull request #34212: [SPARK-36402][PYTHON] Implement Series.combine

[GitBox <gi...@apache.org>]

itholic commented on a change in pull request #34212:
URL: https://github.com/apache/spark/pull/34212#discussion_r769173394



##########
File path: python/pyspark/pandas/series.py
##########
@@ -4485,6 +4489,170 @@ def replace(
 
         return self._with_new_scol(current)  # TODO: dtype?
 
+    def combine(
+        self,
+        other: "Series",
+        func: Callable,
+        fill_value: Optional[Any] = None,
+    ) -> "Series":
+        """
+        Combine the Series with a Series or scalar according to `func`.
+
+        Combine the Series and `other` using `func` to perform elementwise
+        selection for combined Series.
+        `fill_value` is assumed when value is missing at some index
+        from one of the two objects being combined.
+
+        .. versionadded:: 3.3.0
+
+        .. note:: this API executes the function once to infer the type which is
+             potentially expensive, for instance, when the dataset is created after
+             aggregations or sorting.
+
+             To avoid this, specify return type in ``func``, for instance, as below:
+
+             >>> def foo(x, y) -> np.int32:
+             ...     return x * y
+
+             pandas-on-Spark uses return type hint and does not try to infer the type.
+
+        Parameters
+        ----------
+        other : Series or scalar
+            The value(s) to be combined with the `Series`.
+        func : function
+            Function that takes two scalars as inputs and returns an element.
+            Note that type hint for return type is required.

Review comment:
       I think it's not "required", but "strongly recommended" ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org