You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ma...@apache.org on 2026/06/30 11:12:00 UTC

(spark) branch branch-4.x updated: [SPARK-48275][SQL][DOC] Improve array_sort default comparator documentation

This is an automated email from the ASF dual-hosted git repository.

MaxGekk pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 12983be04895 [SPARK-48275][SQL][DOC] Improve array_sort default comparator documentation
12983be04895 is described below

commit 12983be048950207e8d0af53686c9c5ec9b51947
Author: Mao Li <63...@users.noreply.github.com>
AuthorDate: Tue Jun 30 13:11:35 2026 +0200

    [SPARK-48275][SQL][DOC] Improve array_sort default comparator documentation
    
    ### What changes were proposed in this pull request?
    
    This PR improves the Scaladoc for `array_sort` by clarifying the behavior of the default comparator, especially for struct elements.
    
    ### Why are the changes needed?
    
    The current documentation says that array elements must be orderable, but does not explain how the default comparator behaves for structs or why arrays of structs with unorderable fields can fail.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Built the affected module locally with:
    
    `./build/mvn -pl sql/api -am -DskipTests package`
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: ChatGPT GPT-5.4 Thinking
    
    Closes #55164 from maoli67660/SPARK-48275.
    
    Lead-authored-by: Mao Li <63...@users.noreply.github.com>
    Co-authored-by: Mao Li <ma...@roku.com>
    Signed-off-by: Max Gekk <ma...@gmail.com>
    (cherry picked from commit 6fb904b8dce56691474e128eab148cd2d0420d33)
    Signed-off-by: Max Gekk <ma...@gmail.com>
---
 sql/api/src/main/scala/org/apache/spark/sql/functions.scala | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sql/api/src/main/scala/org/apache/spark/sql/functions.scala b/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
index 8aea50291cdc..65f266d9d50d 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/functions.scala
@@ -8941,9 +8941,13 @@ object functions {
   def get(column: Column, index: Column): Column = Column.fn("get", column, index)
 
   /**
-   * Sorts the input array in ascending order. The elements of the input array must be orderable.
-   * NaN is greater than any non-NaN elements for double/float type. Null elements will be placed
-   * at the end of the returned array.
+   * Sorts the input array in ascending order. Null elements will be placed at the end of the
+   * returned array. NaN is greater than any non-NaN elements for double/float type.
+   *
+   * The elements of the input array must be orderable. For example, when the array elements are
+   * structs, the default comparator compares the struct fields in schema order. Therefore, all
+   * fields in the struct must be orderable. If the default comparator does not support the input
+   * type, you can specify a custom comparator.
    *
    * @group collection_funcs
    * @since 2.4.0


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org