[GitHub] [incubator-kyuubi] turboFei commented on a diff in pull request #3894: [KYUUBI #3893] [BUG] Fix spark extension: UnspecifiedDistribution does not have default partitioning.

[GitBox <gi...@apache.org>]

turboFei commented on code in PR #3894:
URL: https://github.com/apache/incubator-kyuubi/pull/3894#discussion_r1038998445


##########
extensions/spark/kyuubi-extension-spark-common/src/main/scala/org/apache/kyuubi/sql/KyuubiQueryStagePreparation.scala:
##########
@@ -79,6 +79,8 @@ object InsertShuffleNodeBeforeJoin extends Rule[SparkPlan] {
       child
     case SortExec(_, _, _: QueryStageExec, _) =>
       child
+    case SortExec(_, _, _: AQEShuffleReadExec, _) =>
+      child

Review Comment:
   > Looks like 3.1 doesn't have AQEShuffleReadExec, and the problem currently only appears in 3.3.x. Should I separate InsertShuffleNodeBeforeJoin to different module and only change spark 3.3 module
   
   Maybe you can check the className here?
   ```
   case SortExec(_, _, e, _) if  e.getClass.getName == "org.apache.spark.sql.execution.adaptive.AQEShuffleReadExec" =>
     child
   ```
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org