You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Sevenannn (via GitHub)" <gi...@apache.org> on 2024/09/24 21:20:25 UTC
[PR] Use original value when comparing with dictionary column in unparser [datafusion]
Sevenannn opened a new pull request, #12610:
URL: https://github.com/apache/datafusion/pull/12610
## Which issue does this PR close?
No relevant issue
## Rationale for this change
Datafusion enforces an eager cast of value to dictionary value in the logical plan where a comparison with dictionary value is presented. For example, in the following plan
```SQL
Filter: person_mood.mood_status = CAST(Utf8("happy") AS Dictionary(Int8, Utf8))
```
However, when using unparser to convert the plan back to sql, where the sql will be sent to various query engine (PostgreSQL, MySQL, etc.) the cast is not needed for the value, in the example above the value `happy`, since the raw value can be directly used in SQL engines without casting to an engine specific dictionary type. Therefore, the plan can simply be rewritten into
```SQL
where person_mood.mood_status = 'happy'
```
## What changes are included in this PR?
* Directly pass `inner_expr` instead of `cast inner_expr as dictionary` in the `Expr::Cast` case when the `inner_expr` is a `Value` representing constant and the plan is `cast to dictionary`
* Add a test for the change
## Are these changes tested?
Yes
## Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Use original value when comparing with dictionary column in unparser [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #12610:
URL: https://github.com/apache/datafusion/pull/12610
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Use original value when comparing with dictionary column in unparser [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #12610:
URL: https://github.com/apache/datafusion/pull/12610#discussion_r1774984617
##########
datafusion/sql/src/unparser/expr.rs:
##########
@@ -219,12 +219,24 @@ impl Unparser<'_> {
}
Expr::Cast(Cast { expr, data_type }) => {
let inner_expr = self.expr_to_sql_inner(expr)?;
- Ok(ast::Expr::Cast {
- kind: ast::CastKind::Cast,
- expr: Box::new(inner_expr),
- data_type: self.arrow_dtype_to_ast_dtype(data_type)?,
- format: None,
- })
+ match data_type {
+ DataType::Dictionary(_, _) => match inner_expr {
+ // Dictionary values don't need to be cast to other types when rewritten back to sql
Review Comment:
this is another example of where logical types as discussed in #11513 might make sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org