You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "LucaCappelletti94 (via GitHub)" <gi...@apache.org> on 2026/08/31 10:39:05 UTC
[PR] Parse SQLite boolean PRAGMA values [datafusion-sqlparser-rs]
LucaCappelletti94 opened a new pull request, #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469
`PRAGMA case_sensitive_like = true` was rejected after parsing the boolean, while SQLite aliases such as `ON`, `OFF`, `YES`, and `NO` were rejected even before reaching pragma validation.
It seemed reasonable to turn these into spanned `Value::Boolean` values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Parse SQLite boolean PRAGMA values [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469#discussion_r3946615141
##########
src/parser/mod.rs:
##########
@@ -20167,15 +20167,34 @@ impl<'a> Parser<'a> {
}
fn parse_pragma_value(&mut self) -> Result<ValueWithSpan, ParserError> {
- let v = self.parse_value()?;
+ let span = self.peek_token_ref().span;
+ let v = match self.parse_one_of_keywords(&[
+ Keyword::TRUE,
+ Keyword::FALSE,
+ Keyword::ON,
+ Keyword::OFF,
+ Keyword::YES,
+ Keyword::NO,
+ ]) {
+ Some(keyword) => Value::Boolean(matches!(
+ keyword,
+ Keyword::TRUE | Keyword::YES | Keyword::ON
+ ))
+ .with_span(span),
+ None => self.parse_value()?,
+ };
match &v.value {
Value::SingleQuotedString(_) => Ok(v),
Value::DoubleQuotedString(_) => Ok(v),
Value::Number(_, _) => Ok(v),
+ Value::Boolean(_) => Ok(v),
Value::Placeholder(_) => Ok(v),
_ => {
self.prev_token();
- self.expected_ref("number or string or ? placeholder", self.peek_token_ref())
+ self.expected_ref(
+ "boolean, number, string, or ? placeholder",
+ self.peek_token_ref(),
+ )
Review Comment:
I think we could simplify the logic to have it accept whatever parse_value returns. and keep the sqlite semantics for validation upstream? that would have avoided this bug in the initial version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] SQLite: parse the full PRAGMA value grammar [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469#discussion_r3966602833
##########
src/parser/mod.rs:
##########
@@ -20157,44 +20157,37 @@ impl<'a> Parser<'a> {
})
}
- fn parse_pragma_value(&mut self) -> Result<ValueWithSpan, ParserError> {
- let v = self.parse_value()?;
- match &v.value {
- Value::SingleQuotedString(_) => Ok(v),
- Value::DoubleQuotedString(_) => Ok(v),
- Value::Number(_, _) => Ok(v),
- Value::Placeholder(_) => Ok(v),
- _ => {
- self.prev_token();
- self.expected_ref("number or string or ? placeholder", self.peek_token_ref())
- }
+ /// Parse a SQLite `pragma-value`: `signed-number | name | signed-literal`.
+ fn parse_pragma_value(&mut self) -> Result<Expr, ParserError> {
+ if matches!(self.peek_token_ref().token, Token::Plus | Token::Minus) {
+ let op = match self.next_token().token {
+ Token::Plus => UnaryOperator::Plus,
+ _ => UnaryOperator::Minus,
+ };
+ return Ok(Expr::UnaryOp {
+ op,
+ expr: Box::new(Expr::Value(self.parse_value()?)),
+ });
+ }
+ match self.maybe_parse(|parser| parser.parse_value())? {
+ Some(value) => Ok(Expr::Value(value)),
+ None => Ok(Expr::Identifier(self.parse_identifier()?)),
}
Review Comment:
In terms of accepting things, yes it would work, but `parse_expr()` would over-extend the grammar accepting arbitrary expressions like `PRAGMA p = 1 + 2`, `foo(bar)`, `a AND b` or `(SELECT 1)`.
SQLite defines a pragma value as `signed-number | name | signed-literal`, so `parse_pragma_value` mirrors that production exactly while still leaving the per-pragma value validation to execution.
I understand I tend to lean much more strongly on the strict parser approach, but here I really believe allowing any expr would be excessively loose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] SQLite: parse the full PRAGMA value grammar [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469#discussion_r3966525366
##########
src/parser/mod.rs:
##########
@@ -20157,44 +20157,37 @@ impl<'a> Parser<'a> {
})
}
- fn parse_pragma_value(&mut self) -> Result<ValueWithSpan, ParserError> {
- let v = self.parse_value()?;
- match &v.value {
- Value::SingleQuotedString(_) => Ok(v),
- Value::DoubleQuotedString(_) => Ok(v),
- Value::Number(_, _) => Ok(v),
- Value::Placeholder(_) => Ok(v),
- _ => {
- self.prev_token();
- self.expected_ref("number or string or ? placeholder", self.peek_token_ref())
- }
+ /// Parse a SQLite `pragma-value`: `signed-number | name | signed-literal`.
+ fn parse_pragma_value(&mut self) -> Result<Expr, ParserError> {
+ if matches!(self.peek_token_ref().token, Token::Plus | Token::Minus) {
+ let op = match self.next_token().token {
+ Token::Plus => UnaryOperator::Plus,
+ _ => UnaryOperator::Minus,
+ };
+ return Ok(Expr::UnaryOp {
+ op,
+ expr: Box::new(Expr::Value(self.parse_value()?)),
+ });
+ }
+ match self.maybe_parse(|parser| parser.parse_value())? {
+ Some(value) => Ok(Expr::Value(value)),
+ None => Ok(Expr::Identifier(self.parse_identifier()?)),
}
Review Comment:
hmm can this be replaced by a call to `parse_expr()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] SQLite: parse the full PRAGMA value grammar [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on code in PR #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469#discussion_r3946785517
##########
src/parser/mod.rs:
##########
@@ -20167,15 +20167,34 @@ impl<'a> Parser<'a> {
}
fn parse_pragma_value(&mut self) -> Result<ValueWithSpan, ParserError> {
- let v = self.parse_value()?;
+ let span = self.peek_token_ref().span;
+ let v = match self.parse_one_of_keywords(&[
+ Keyword::TRUE,
+ Keyword::FALSE,
+ Keyword::ON,
+ Keyword::OFF,
+ Keyword::YES,
+ Keyword::NO,
+ ]) {
+ Some(keyword) => Value::Boolean(matches!(
+ keyword,
+ Keyword::TRUE | Keyword::YES | Keyword::ON
+ ))
+ .with_span(span),
+ None => self.parse_value()?,
+ };
match &v.value {
Value::SingleQuotedString(_) => Ok(v),
Value::DoubleQuotedString(_) => Ok(v),
Value::Number(_, _) => Ok(v),
+ Value::Boolean(_) => Ok(v),
Value::Placeholder(_) => Ok(v),
_ => {
self.prev_token();
- self.expected_ref("number or string or ? placeholder", self.peek_token_ref())
+ self.expected_ref(
+ "boolean, number, string, or ? placeholder",
+ self.peek_token_ref(),
+ )
Review Comment:
I checked SQLite docs hoping for a limited set of allowed values, it states there is explicitly no such set, and I therefore switched to `Expr`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] SQLite: parse the full PRAGMA value grammar [datafusion-sqlparser-rs]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #2469:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2469#issuecomment-5823450706
## [Codecov](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/pull/2469?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `75.00000%` with `4 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 81.01%. Comparing base ([`88ffee6`](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/commit/88ffee641a2192f17b44fec658463e2963233ec3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`490a61a`](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/commit/490a61a0446ba9f4caee0ab8f502b1df998818ea?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/pull/2469?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [src/parser/mod.rs](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/pull/2469?src=pr&el=tree&filepath=src%2Fparser%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3JjL3BhcnNlci9tb2QucnM=) | 75.00% | [0 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/pull/2469?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #2469 +/- ##
==========================================
- Coverage 81.02% 81.01% -0.01%
==========================================
Files 42 42
Lines 33436 33430 -6
Branches 33436 33430 -6
==========================================
- Hits 27090 27084 -6
+ Misses 2789 2787 -2
- Partials 3557 3559 +2
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion-sqlparser-rs/pull/2469?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org