You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/08/20 10:57:09 UTC
[GitHub] [arrow-datafusion] alamb commented on issue #533: Add extension plugin to parse SQL into logical plan / user defined SQL
alamb commented on issue #533:
URL: https://github.com/apache/arrow-datafusion/issues/533#issuecomment-1221293210
An update here:
There are several examples of people who have implemented their own custom query language / domain specific language using DataFusion. For example vega fusion with the vega language, and InfluxDB IOx with the language that underlies flux and influxql.
Typically the approach is to use the `LogialPlanBuilder` and skip the datafusion SQLPLanner entirely
One way we might be able to accomplish this extension point would be like:
```rust
/// Default SQL planner
pub trait SQLPlanner :
fn plan_query(&mut self, query: sqlparser::ast::Query) -> LogicalPlan {
// call free function
plan_query(self, query)
}
fn plan_expr(&mut self, expr: sqlparser::ast::Expr) -> Expr {
// existing logic work
plan_expr(self, expr)
}
}
// Free functions that do the work (as you can't call default trait impls)
// https://stackoverflow.com/questions/43849041/reuse-default-method-implementations-in-trait-impls
fn plan_query<P: SQLPlanner>(planner: &mut P, query: sqlparser::ast::Query) -> LogicalPlan {
// existing logic
}
fn plan_expr<P: SQLPlanner>(planner: &mut P, expr: sqlparser::ast::Expr) -> Expr {
// existing logic work
}
```
I was able to make this type of pattern work for "sql rewriting" where the user is allowed to rewrite an SQL query prior to planning with the existing datafusion planner. You can see a sketch of such a solution in https://github.com/influxdata/influxdb_iox/pull/5438.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org