You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/07 07:33:09 UTC
[GitHub] [spark] beliefer opened a new pull request #35746: [WIP][SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way
beliefer opened a new pull request #35746:
URL: https://github.com/apache/spark/pull/35746
### What changes were proposed in this pull request?
Currently, Spark DS V2 could push down filters into JDBC source. However, only the most basic form of filter is supported.
On the other hand, some JDBC source could not compile the filters by themselves way.
This PR reactor the framework so as JDBC dialect could compile expression by self way.
First, The framework translate catalyst expressions to DS V2 filters.
Second, The JDBC dialect could compile DS V2 filters to different SQL syntax.
### Why are the changes needed?
Make the framework be more common use.
### Does this PR introduce _any_ user-facing change?
'No'.
The feature is not released.
### How was this patch tested?
Exists tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer closed pull request #35746: [WIP][SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way
Posted by GitBox <gi...@apache.org>.
beliefer closed pull request #35746:
URL: https://github.com/apache/spark/pull/35746
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #35746: [WIP][SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #35746:
URL: https://github.com/apache/spark/pull/35746#issuecomment-1060785799
It seems like we can completely replace v2 `Filter` with `GeneralScalarExpression`, as they are both tree-like API to represent catalyst expressions in DS v2.
For better type safety, I think we can make `Filter` extends `GeneralScalarExpression`, to indicate that it's a predicate that returns boolean.
That said, the `V2ExpressionBuilder` should create `Filter` if the catalyst expression is a predicate (e.g. binary comparison).
also cc @huaxingao for more opinions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #35746: [WIP][SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #35746:
URL: https://github.com/apache/spark/pull/35746#issuecomment-1066032475
https://github.com/apache/spark/pull/35768 replace this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #35746: [WIP][SPARK-38432][SQL] Reactor framework so as JDBC dialect could compile filter by self way
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #35746:
URL: https://github.com/apache/spark/pull/35746#issuecomment-1060788234
BTW `GeneralScalarExpression` should overwrite `equals` and `hashCode`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org