You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/06/10 10:24:48 UTC
[GitHub] [spark] beliefer opened a new pull request, #36830: [SPARK-38761][SQL][FOLLOWUP] DS V2 supports push down misc non-aggregate functions
beliefer opened a new pull request, #36830:
URL: https://github.com/apache/spark/pull/36830
### What changes were proposed in this pull request?
https://github.com/apache/spark/pull/36039 makes DS V2 supports push down misc non-aggregate functions are claimed by ANSI standard.
Spark have a lot common used misc non-aggregate functions are not claimed by ANSI standard.
https://github.com/apache/spark/blob/2f8613f22c0750c00cf1dcfb2f31c431d8dc1be7/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala#L362.
The mainstream databases support these functions show below.
| Function name | PostgreSQL | ClickHouse | H2 | MySQL | Oracle | Redshift | Presto | Teradata | Snowflake | DB2 | Vertica | Exasol | SqlServer | Yellowbrick | Impala | Mariadb | Druid | Pig | Singlestore | ElasticSearch | SQLite | Influxdata | Sybase |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| `GREATEST` | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | No | No |
| `LEAST` | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | No | No |
| `IF` | No | Yes | No | Yes | No | No | Yes | No | Yes | No | No | Yes | No | Yes | Yes | Yes | No | No | Yes | Yes | Yes | No | No |
| `RAND` | No | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | No | Yes |
### Why are the changes needed?
DS V2 supports push down misc non-aggregate functions supported by mainstream databases.
### Does this PR introduce _any_ user-facing change?
'No'.
New feature.
### How was this patch tested?
New tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r902337504
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/GeneralScalarExpression.java:
##########
@@ -106,6 +106,30 @@
* <li>Since version: 3.3.0</li>
* </ul>
* </li>
+ * <li>Name: <code>GREATEST</code>
+ * <ul>
+ * <li>SQL semantic: <code>GREATEST(expr, ...)</code></li>
+ * <li>Since version: 3.4.0</li>
+ * </ul>
+ * </li>
+ * <li>Name: <code>LEAST</code>
+ * <ul>
+ * <li>SQL semantic: <code>LEAST(expr, ...)</code></li>
+ * <li>Since version: 3.4.0</li>
+ * </ul>
+ * </li>
+ * <li>Name: <code>IF</code>
+ * <ul>
+ * <li>SQL semantic: <code>IF(expr1, expr2, expr3)</code></li>
Review Comment:
We can rewrite IF to CASE WHEN and push it down?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r902448791
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/GeneralScalarExpression.java:
##########
@@ -106,6 +106,30 @@
* <li>Since version: 3.3.0</li>
* </ul>
* </li>
+ * <li>Name: <code>GREATEST</code>
+ * <ul>
+ * <li>SQL semantic: <code>GREATEST(expr, ...)</code></li>
+ * <li>Since version: 3.4.0</li>
+ * </ul>
+ * </li>
+ * <li>Name: <code>LEAST</code>
+ * <ul>
+ * <li>SQL semantic: <code>LEAST(expr, ...)</code></li>
+ * <li>Since version: 3.4.0</li>
+ * </ul>
+ * </li>
+ * <li>Name: <code>IF</code>
+ * <ul>
+ * <li>SQL semantic: <code>IF(expr1, expr2, expr3)</code></li>
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r910878016
##########
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala:
##########
@@ -425,6 +425,26 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession with ExplainSuiteHel
checkFiltersRemoved(df10)
checkPushedInfo(df10, "PushedFilters: [ID IS NOT NULL, ID > 1], ")
checkAnswer(df10, Row("mary", 2))
+
+ val df11 = sql(
+ """
+ |SELECT * FROM h2.test.employee
+ |WHERE GREATEST(bonus, 1100) > 1200 AND LEAST(salary, 10000) > 9000 AND RAND(1) < 1
Review Comment:
the seed is given, so result is still stable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
huaxingao commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1154634939
So the statement here https://github.com/apache/spark/pull/36039#issuecomment-1089567136 is not true any more, we will push down both ANSI functions and commonly used non-ANSI functions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1165388053
ping @cloud-fan @huaxingao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r910866940
##########
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala:
##########
@@ -425,6 +425,26 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession with ExplainSuiteHel
checkFiltersRemoved(df10)
checkPushedInfo(df10, "PushedFilters: [ID IS NOT NULL, ID > 1], ")
checkAnswer(df10, Row("mary", 2))
+
+ val df11 = sql(
+ """
+ |SELECT * FROM h2.test.employee
+ |WHERE GREATEST(bonus, 1100) > 1200 AND LEAST(salary, 10000) > 9000 AND RAND(1) < 1
Review Comment:
If so, the results will be unstable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1154641204
> So the statement here [#36039 (comment)](https://github.com/apache/spark/pull/36039#issuecomment-1089567136) is not true any more, we will push down both ANSI functions and commonly used non-ANSI functions?
Yes. The ANSI function too few and the commonly used non-ANSI functions also needed.
A discussion between @cloud-fan and me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1155894171
> Can we add documentation for function push down?
Oh. I forgot it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
huaxingao commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1155811468
Can we add documentation for function push down?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] huaxingao commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
huaxingao commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r897413352
##########
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala:
##########
@@ -424,6 +424,24 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession with ExplainSuiteHel
checkFiltersRemoved(df10)
checkPushedInfo(df10, "PushedFilters: [ID IS NOT NULL, ID > 1], ")
checkAnswer(df10, Row("mary", 2))
+
+ val df11 = sql(
+ """
+ |SELECT * FROM h2.test.employee
+ |WHERE GREATEST(bonus, 1100) > 1200 AND LEAST(salary, 10000) > 9000 AND RAND(1) < 1
+ |""".stripMargin)
+ checkFiltersRemoved(df11)
+ checkPushedInfo(df11, "PushedFilters: " +
+ "[(GREATEST(BONUS, 1100.0)) > 1200.0, (LEAST(SALARY, 10000.00)) > 9000.00, RAND(1) < 1.0]")
+ checkAnswer(df11, Row(2, "david", 10000, 1300, true))
+
+ val df12 = sql("""
Review Comment:
nit: can we follow the style at line 428 for consistency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #36830: [SPARK-38761][SQL][FOLLOWUP] DS V2 supports push down misc non-aggregate functions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r895264380
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java:
##########
@@ -97,6 +97,10 @@ public String build(Expression expr) {
return visitUnaryArithmetic(name, inputToSQL(e.children()[0]));
case "ABS":
case "COALESCE":
+ case "GREATEST":
+ case "LEAST":
+ case "IF":
+ case "RAND":
Review Comment:
Let's file a separate JIRA if it adds more supports of other stuff
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
URL: https://github.com/apache/spark/pull/36830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1165425089
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #36830: [SPARK-38761][SQL][FOLLOWUP] DS V2 supports push down misc non-aggregate functions
Posted by GitBox <gi...@apache.org>.
beliefer commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r895270044
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java:
##########
@@ -97,6 +97,10 @@ public String build(Expression expr) {
return visitUnaryArithmetic(name, inputToSQL(e.children()[0]));
case "ABS":
case "COALESCE":
+ case "GREATEST":
+ case "LEAST":
+ case "IF":
+ case "RAND":
Review Comment:
Good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1165486661
@cloud-fan @huaxingao @HyukjinKwon Thank you for you review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r897483927
##########
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala:
##########
@@ -424,6 +424,24 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession with ExplainSuiteHel
checkFiltersRemoved(df10)
checkPushedInfo(df10, "PushedFilters: [ID IS NOT NULL, ID > 1], ")
checkAnswer(df10, Row("mary", 2))
+
+ val df11 = sql(
+ """
+ |SELECT * FROM h2.test.employee
+ |WHERE GREATEST(bonus, 1100) > 1200 AND LEAST(salary, 10000) > 9000 AND RAND(1) < 1
+ |""".stripMargin)
+ checkFiltersRemoved(df11)
+ checkPushedInfo(df11, "PushedFilters: " +
+ "[(GREATEST(BONUS, 1100.0)) > 1200.0, (LEAST(SALARY, 10000.00)) > 9000.00, RAND(1) < 1.0]")
+ checkAnswer(df11, Row(2, "david", 10000, 1300, true))
+
+ val df12 = sql("""
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r910841713
##########
sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCV2Suite.scala:
##########
@@ -425,6 +425,26 @@ class JDBCV2Suite extends QueryTest with SharedSparkSession with ExplainSuiteHel
checkFiltersRemoved(df10)
checkPushedInfo(df10, "PushedFilters: [ID IS NOT NULL, ID > 1], ")
checkAnswer(df10, Row("mary", 2))
+
+ val df11 = sql(
+ """
+ |SELECT * FROM h2.test.employee
+ |WHERE GREATEST(bonus, 1100) > 1200 AND LEAST(salary, 10000) > 9000 AND RAND(1) < 1
Review Comment:
`RAND(1) < 1` is always true, which is bad as a filter pushdown test. Can we change it to `RAND(1) > 0.5`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #36830: [SPARK-39453][SQL] DS V2 supports push down misc non-aggregate functions(non ANSI)
Posted by GitBox <gi...@apache.org>.
beliefer commented on PR #36830:
URL: https://github.com/apache/spark/pull/36830#issuecomment-1153565155
ping @huaxingao cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a diff in pull request #36830: [SPARK-38761][SQL][FOLLOWUP] DS V2 supports push down misc non-aggregate functions
Posted by GitBox <gi...@apache.org>.
beliefer commented on code in PR #36830:
URL: https://github.com/apache/spark/pull/36830#discussion_r895270044
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java:
##########
@@ -97,6 +97,10 @@ public String build(Expression expr) {
return visitUnaryArithmetic(name, inputToSQL(e.children()[0]));
case "ABS":
case "COALESCE":
+ case "GREATEST":
+ case "LEAST":
+ case "IF":
+ case "RAND":
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org