Re: [PR] chore: Improve reporting of fallback reasons for CollectLimit [datafusion-comet]

["andygrove (via GitHub)" <gi...@apache.org>]

andygrove commented on code in PR #1694:
URL: https://github.com/apache/datafusion-comet/pull/1694#discussion_r2162109484


##########
spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala:
##########
@@ -196,18 +198,34 @@ case class CometExecRule(session: SparkSession) extends Rule[SparkPlan] {
           op,
           CometGlobalLimitExec(_, op, op.limit, op.child, SerializedPlan(None)))
 
-      case op: CollectLimitExec
-          if isCometNative(op.child) && CometConf.COMET_EXEC_COLLECT_LIMIT_ENABLED.get(conf)
-            && isCometShuffleEnabled(conf)
-            && op.offset == 0 =>
-        QueryPlanSerde
-          .operator2Proto(op)
-          .map { nativeOp =>
-            val cometOp =
-              CometCollectLimitExec(op, op.limit, op.offset, op.child)
-            CometSinkPlaceHolder(nativeOp, op, cometOp)
+      case op: CollectLimitExec =>
+        val fallbackReasons = new ListBuffer[String]()
+        if (!CometConf.COMET_EXEC_COLLECT_LIMIT_ENABLED.get(conf)) {
+          fallbackReasons += s"${CometConf.COMET_EXEC_COLLECT_LIMIT_ENABLED.key} is false"
+        }
+        if (!isCometShuffleEnabled(conf)) {
+          fallbackReasons += "Comet shuffle is not enabled"
+        }
+        if (op.offset != 0) {
+          fallbackReasons += "CollectLimit with non-zero offset is not supported"
+        }
+        if (fallbackReasons.nonEmpty) {
+          withInfos(op, fallbackReasons.toSet)
+        } else {
+          if (!isCometNative(op.child)) {
+            // no reason to report reason if child is not native
+            op

Review Comment:
   Yes, that is correct.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org