You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ConeyLiu (via GitHub)" <gi...@apache.org> on 2023/08/25 10:03:53 UTC
[GitHub] [iceberg] ConeyLiu commented on a diff in pull request #8394: Spark 3.4: Rewrite procedure throw better exception when filter expression cannot translate
ConeyLiu commented on code in PR #8394:
URL: https://github.com/apache/iceberg/pull/8394#discussion_r1305463982
##########
spark/v3.4/spark/src/main/scala/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala:
##########
@@ -35,7 +35,14 @@ object SparkExpressionConverter {
// Currently, it is a double conversion as we are converting Spark expression to Spark filter
// and then converting Spark filter to Iceberg expression.
// But these two conversions already exist and well tested. So, we are going with this approach.
- SparkFilters.convert(DataSourceStrategy.translateFilter(sparkExpression, supportNestedPredicatePushdown = true).get)
+ DataSourceStrategy.translateFilter(sparkExpression, supportNestedPredicatePushdown = true) match {
Review Comment:
We should change this to V2 translate and V2 filter. Then we could convert the system functions to Iceberg expression after #8088 or after https://github.com/apache/spark/pull/42612.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org