[GitHub] [spark] HeartSaVioR commented on a change in pull request #35673: [SPARK-38204][SS] Use StatefulOpClusteredDistribution for stateful operators with respecting backward compatibility

[GitBox <gi...@apache.org>]

HeartSaVioR commented on a change in pull request #35673:
URL: https://github.com/apache/spark/pull/35673#discussion_r815567106



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala
##########
@@ -696,11 +687,8 @@ case class SessionWindowStateStoreSaveExec(
   override def outputPartitioning: Partitioning = child.outputPartitioning
 
   override def requiredChildDistribution: Seq[Distribution] = {
-    // NOTE: Please read through the NOTE on the classdoc of StatefulOpClusteredDistribution
-    // before making any changes.
-    // TODO(SPARK-38204)
-    ClusteredDistribution(keyExpressions,
-      requiredNumPartitions = stateInfo.map(_.numPartitions)) :: Nil
+    StatefulOperatorPartitioning.getClusteredDistributionWithBackwardCompatibility(

Review comment:
       I just changed keyExpressions to keyWithoutSessionExpressions which is actually same, but clearer to see the behavior.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org