[GitHub] cloud-fan commented on a change in pull request #23249: [SPARK-26297][SQL] improve the doc of Distribution/Partitioning

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #23249: [SPARK-26297][SQL] improve the doc of Distribution/Partitioning
URL: https://github.com/apache/spark/pull/23249#discussion_r240431621
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala
 ##########
 @@ -118,10 +115,12 @@ case class HashClusteredDistribution(
 
 /**
  * Represents data where tuples have been ordered according to the `ordering`
- * [[Expression Expressions]].  This is a strictly stronger guarantee than
- * [[ClusteredDistribution]] as an ordering will ensure that tuples that share the
- * same value for the ordering expressions are contiguous and will never be split across
- * partitions.
+ * [[Expression Expressions]]. Its requirement is defined as the following:
+ *   - Given any 2 adjacent partitions, all the rows of the second partition must be larger than or
+ *     equal to any row in the first partition, according to the `ordering` expressions.
 
 Review comment:
   @hvanhovell We need this relaxed requirement, otherwise we have to remove the optimization [here](https://github.com/apache/spark/pull/23249/files#diff-14e5db1a500000efdaf9c760aeb5ced4R273)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org