[GitHub] [spark] huaxingao commented on a diff in pull request #36330: [SPARK-38897][SQL] DS V2 supports push down string functions

[GitBox <gi...@apache.org>]

huaxingao commented on code in PR #36330:
URL: https://github.com/apache/spark/pull/36330#discussion_r867651294


##########
sql/core/src/main/scala/org/apache/spark/sql/jdbc/H2Dialect.scala:
##########
@@ -41,6 +41,11 @@ private object H2Dialect extends JdbcDialect {
         case _ => super.visitSQLFunction(funcName, inputs)
       }
     }
+
+    override def visitOverlay(inputs: Array[String]): String = {
+      val functionInfo = super.visitOverlay(inputs)
+      throw QueryCompilationErrors.noSuchFunctionError("H2", functionInfo)

Review Comment:
   For the functions that are not supported by every DBMS, e.g. `overlay` and `width_bucket`, I think the default behavior should be not pushing down and then we enable push down in the dialect if the database supports it. Otherwise, we will need to turn off push down in all the database that don't support this function. Without doing this, we will have inconsistent behaviors before/after this PR.
   For example, `SELECT * FROM ... WHERE OVERLAY ...` will fail for `MySQL` after this PR. I think before this PR it should pass.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org