[GitHub] [spark] cloud-fan commented on pull request #29810: [SPARK-32940][SQL] Collect, first and last should be deterministic aggregate functions

[GitBox <gi...@apache.org>]

cloud-fan commented on pull request #29810:
URL: https://github.com/apache/spark/pull/29810#issuecomment-961268006


   There is inevitable randomness in the input of aggregate functions, because the shuffle reader may produce data with random orders, and we are not able to completely eliminate this randomness. For example, even `sum` has randomness, as adding up floating values with different orders can lead to slightly different results.
   
   I don't think `first/last/collect` has a significant difference and +1 to mark them as deterministic.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org