[GitHub] [spark] cloud-fan commented on a change in pull request #35041: [SPARK-37691][SQL] Support ANSI Aggregation Function: `percentile_disc`

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #35041:
URL: https://github.com/apache/spark/pull/35041#discussion_r821440019



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/percentiles.scala
##########
@@ -324,3 +337,117 @@ case class Percentile(
     frequencyExpression = newThird
   )
 }
+
+/**
+ * The Percentile aggregate function returns the percentile(s) based on a discrete distribution of
+ * numeric column `expr` at the given percentage(s) with value range in [0.0, 1.0].
+ *
+ * Because the number of elements and their partial order cannot be determined in advance.
+ * Therefore we have to store all the elements in memory, and so notice that too many elements can
+ * cause GC paused and eventually OutOfMemory Errors.
+ */

Review comment:
       We need to write SQL function doc for it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org