You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/03/07 18:42:08 UTC
[I] Upgrade to sqlparser 0.55.0 [datafusion]
alamb opened a new issue, #15071:
URL: https://github.com/apache/datafusion/issues/15071
### Is your feature request related to a problem or challenge?
The new version of sqlparser has been released:
https://crates.io/crates/sqlparser/0.55.0
Datafusion is still on sqlparser 0.54.0: https://github.com/apache/datafusion/blob/24586938518d747c16dd0702454fb334ab426a85/Cargo.toml#L164
### Describe the solution you'd like
I would like to upgrade datafusion to 0.55.0 of sqlparser
This one may be tricky due to the change in ObjectName
### Describe alternatives you've considered
Here is an example showing how it was done last time
- https://github.com/apache/datafusion/pull/14255
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "PokIsemaine (via GitHub)" <gi...@apache.org>.
PokIsemaine commented on issue #15071:
URL: https://github.com/apache/datafusion/issues/15071#issuecomment-2707977800
@alamb @jonahgao
I have already made some progress on this part. If you don't mind, I would like to finish it. I want to use this update to become more familiar with DataFusion's code, and I would greatly appreciate any guidance you can offer.
Here are some sqlparser Pull Requests that involve modifications to DataFusion, which you can use as references:
Changelog: https://github.com/apache/datafusion-sqlparser-rs/blob/main/changelog/0.55.0.md
OrderBy: https://github.com/apache/datafusion-sqlparser-rs/pull/1724
SQLExpr::Case: https://github.com/apache/datafusion-sqlparser-rs/pull/1733
ColumnDef Collation: https://github.com/apache/datafusion-sqlparser-rs/pull/1731
DataType: https://github.com/apache/datafusion-sqlparser-rs/pull/1739
EXECUTE: https://github.com/apache/datafusion-sqlparser-rs/pull/1717
CreateTable: https://github.com/apache/datafusion-sqlparser-rs/pull/1664
QualifiedWildcard https://github.com/apache/datafusion-sqlparser-rs/pull/1680
TypedString: https://github.com/apache/datafusion-sqlparser-rs/pull/1739
SelectFlavor: https://github.com/apache/datafusion-sqlparser-rs/pull/1713
StartTransaction: https://github.com/apache/datafusion-sqlparser-rs/pull/1718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao commented on issue #15071:
URL: https://github.com/apache/datafusion/issues/15071#issuecomment-2708037360
Thank you @PokIsemaine for your work. Assigned this issue to you ❤️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao closed issue #15071: Upgrade to sqlparser 0.55.0
URL: https://github.com/apache/datafusion/issues/15071
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao commented on issue #15071:
URL: https://github.com/apache/datafusion/issues/15071#issuecomment-2708885625
> Do we need to distinguish between `Inner/Join`, `Left/LeftOuter`, and `Right/RightOuter` in `JoinType`, like this?
I think we don't need to distinguish them as they are equivalent. Introducing new join type variants would result in more significant changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "PokIsemaine (via GitHub)" <gi...@apache.org>.
PokIsemaine commented on issue #15071:
URL: https://github.com/apache/datafusion/issues/15071#issuecomment-2708645397
I want to discuss `JoinType`:
JoinOperator
- https://github.com/apache/datafusion-sqlparser-rs/pull/1692
- https://github.com/apache/datafusion-sqlparser-rs/pull/1726
Do we need to distinguish between `Inner/Join`, `Left/LeftOuter`, and `Right/RightOuter` in `JoinType`, like this?
```rust
pub enum JoinType {
Inner,
Join,
Left,
LeftOuter,
Right,
RightOuter,
// other...
}
```
Since `sqlparser` now differentiates between `INNER JOIN` and `JOIN`, the parsed result no longer matches the original test cases. For example:
```rust
TestStatementWithDialect {
sql: "
SELECT
j1_string,
j2_string
FROM
(
SELECT
distinct j1_id,
j1_string,
j2_string
from
j1
INNER join j2 ON j1.j1_id = j2.j2_id
order by
j1.j1_id desc
limit
10
) abc
ORDER BY
abc.j2_string",
expected: r#"SELECT abc.j1_string, abc.j2_string FROM (SELECT DISTINCT j1.j1_id, j1.j1_string, j2.j2_string FROM j1 JOIN j2 ON (j1.j1_id = j2.j2_id) ORDER BY j1.j1_id DESC NULLS FIRST LIMIT 10) AS abc ORDER BY abc.j2_string ASC NULLS LAST"#,
parser_dialect: Box::new(GenericDialect {}),
unparser_dialect: Box::new(UnparserDefaultDialect {}),
},
```
Should the `JOIN` in `expected` be changed to `INNER JOIN`?
If we make this distinction, it will affect printing and the unparser, which will indirectly impact many test cases. Do you have any good suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Upgrade to sqlparser 0.55.0 [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao commented on issue #15071:
URL: https://github.com/apache/datafusion/issues/15071#issuecomment-2707862584
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org