Re: [PR] [SPARK-47007][SQL][PYTHON][R][CONNECT] map_sort function [spark]

["MaxGekk (via GitHub)" <gi...@apache.org>]

MaxGekk commented on code in PR #45069:
URL: https://github.com/apache/spark/pull/45069#discussion_r1530661898


##########
python/pyspark/sql/functions/builtin.py:
##########
@@ -16840,6 +16840,54 @@ def map_concat(
     return _invoke_function_over_seq_of_columns("map_concat", cols)  # type: ignore[arg-type]
 
 
+@_try_remote_functions
+def map_sort(col: "ColumnOrName", asc: bool = True) -> Column:
+    """
+    Map function: Sorts the input map in ascending or descending order according
+    to the natural ordering of the map keys.
+
+    .. versionadded:: 4.0.0
+
+    Parameters
+    ----------
+    col : :class:`~pyspark.sql.Column` or str
+        Name of the column or expression.
+    asc : bool, optional
+        Whether to sort in ascending or descending order. If `asc` is True (default),
+        then the sorting is in ascending order. If False, then in descending order.
+
+    Returns
+    -------
+    :class:`~pyspark.sql.Column`
+        Sorted map.
+
+    Examples
+    --------
+    Example 1: Sorting a map in ascending order
+
+    >>> import pyspark.sql.functions as sf
+    >>> df = spark.sql("SELECT map(3, 'c', 1, 'a', 2, 'b') as data")
+    >>> df.select(sf.map_sort(df.data)).show(truncate=False)
+    +--------------------+
+    |map_sort(data, true)|
+    +--------------------+
+    |{1 -> a, 2 -> b, ...|

Review Comment:
   If `truncate` = `False`, why it is truncated?
   
   I have double checked locally:
   ```python
   >>> df.select(sf.map_sort(df.data)).show(truncate=False)
   +------------------------+
   |map_sort(data, true)    |
   +------------------------+
   |{1 -> a, 2 -> b, 3 -> c}|
   +------------------------+
   ```



##########
python/pyspark/sql/functions/builtin.py:
##########
@@ -16840,6 +16840,54 @@ def map_concat(
     return _invoke_function_over_seq_of_columns("map_concat", cols)  # type: ignore[arg-type]
 
 
+@_try_remote_functions
+def map_sort(col: "ColumnOrName", asc: bool = True) -> Column:
+    """
+    Map function: Sorts the input map in ascending or descending order according
+    to the natural ordering of the map keys.
+
+    .. versionadded:: 4.0.0
+
+    Parameters
+    ----------
+    col : :class:`~pyspark.sql.Column` or str
+        Name of the column or expression.
+    asc : bool, optional
+        Whether to sort in ascending or descending order. If `asc` is True (default),
+        then the sorting is in ascending order. If False, then in descending order.
+
+    Returns
+    -------
+    :class:`~pyspark.sql.Column`
+        Sorted map.
+
+    Examples
+    --------
+    Example 1: Sorting a map in ascending order
+
+    >>> import pyspark.sql.functions as sf
+    >>> df = spark.sql("SELECT map(3, 'c', 1, 'a', 2, 'b') as data")
+    >>> df.select(sf.map_sort(df.data)).show(truncate=False)
+    +--------------------+
+    |map_sort(data, true)|
+    +--------------------+
+    |{1 -> a, 2 -> b, ...|
+    +--------------------+
+
+    Example 2: Sorting a map in descending order
+
+    >>> import pyspark.sql.functions as sf
+    >>> df = spark.sql("SELECT map(3, 'c', 1, 'a', 2, 'b') as data")
+    >>> df.select(sf.map_sort(df.data, False)).show(truncate=False)
+    +---------------------+
+    |map_sort(data, false)|
+    +---------------------+
+    | {3 -> c, 2 -> b, ...|

Review Comment:
   Probably, need to fix the output.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org