You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "xitep (via GitHub)" <gi...@apache.org> on 2026/01/14 09:20:24 UTC
[PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
xitep opened a new pull request, #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#discussion_r2726915136
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
+ if !supports_multiline && !supports_singleline {
+ return Ok(None);
+ }
+ loop {
+ let t = self.peek_nth_token_no_skip_ref(0);
+ match &t.token {
+ // ~ only the very first comment
+ Token::Whitespace(ws) => {
+ match ws {
+ Whitespace::SingleLineComment { comment, prefix } => {
+ return Ok(if supports_singleline && comment.starts_with("+") {
+ let text = comment.split_at(1).1.into();
+ let prefix = prefix.clone();
+ self.next_token_no_skip(); // ~ consume the token
+ Some(OptimizerHint {
+ text,
+ style: OptimizerHintStyle::SingleLine { prefix },
+ })
+ } else {
+ None
+ });
+ }
+ Whitespace::MultiLineComment(comment) => {
+ return Ok(if supports_multiline && comment.starts_with("+") {
+ let text = comment.split_at(1).1.into();
+ self.next_token_no_skip(); // ~ consume the token
+ Some(OptimizerHint {
+ text,
+ style: OptimizerHintStyle::MultiLine,
+ })
+ } else {
+ None
+ });
+ }
+ // ~ but skip (pure) whitespace
+ Whitespace::Space | Whitespace::Tab | Whitespace::Newline => {
+ // ~ consume the token and try with the next whitespace (if any)
Review Comment:
```suggestion
// but skip (pure) whitespace
Whitespace::Space | Whitespace::Tab | Whitespace::Newline => {
// consume the token and try with the next whitespace (if any)
```
let's avoid the `~` before the comments, since that's not a convention used in the codebase
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
+ if !supports_multiline && !supports_singleline {
+ return Ok(None);
+ }
+ loop {
+ let t = self.peek_nth_token_no_skip_ref(0);
+ match &t.token {
+ // ~ only the very first comment
Review Comment:
hmm not sure I followed, what does this comment mean by 'very first comment'?
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
Review Comment:
```suggestion
fn maybe_parse_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
```
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
+ if !supports_multiline && !supports_singleline {
+ return Ok(None);
+ }
+ loop {
+ let t = self.peek_nth_token_no_skip_ref(0);
+ match &t.token {
+ // ~ only the very first comment
+ Token::Whitespace(ws) => {
+ match ws {
+ Whitespace::SingleLineComment { comment, prefix } => {
+ return Ok(if supports_singleline && comment.starts_with("+") {
+ let text = comment.split_at(1).1.into();
+ let prefix = prefix.clone();
+ self.next_token_no_skip(); // ~ consume the token
Review Comment:
```suggestion
self.next_token_no_skip(); // Consume the comment token
```
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
Review Comment:
can we change this to use dialect methods? e.g. `if self.supports_comment_optimizer_hint()`
Also I think we can use the same flag and let the parser always accept either the single or multiline cases if they show up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "xitep (via GitHub)" <gi...@apache.org>.
xitep commented on code in PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#discussion_r2727049681
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
Review Comment:
i'm not sure `-- + foo` is a valid hint for mysql (that's why i did the separation in the first place.) on oracle a `--+ quux` is valid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "xitep (via GitHub)" <gi...@apache.org>.
xitep commented on code in PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#discussion_r2727049681
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
Review Comment:
i'm not sure `-- + foo` is a valid hint for mysql (i believe it's not.) that's why i did the separation in the first place. for oracle, though, a `--+ quux` is as valid as `/*+ quux */`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "xitep (via GitHub)" <gi...@apache.org>.
xitep commented on PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#issuecomment-3785289765
surely it would be one way. (the only premise for that would to be have the next non-comment token after the corresponding keywords covered with a span, like suggested in #2099.) however, when processing such statements, this information is some you generally don't want to leave out, even when parsing _without_ comments. hence, this proposal to embed it into the AST explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#issuecomment-3784777785
@xitep wondering would it not make more sense to support features like these via the new `parse_sql_with_comments` API?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "xitep (via GitHub)" <gi...@apache.org>.
xitep commented on code in PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#discussion_r2727019526
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
+ if !supports_multiline && !supports_singleline {
+ return Ok(None);
+ }
+ loop {
+ let t = self.peek_nth_token_no_skip_ref(0);
+ match &t.token {
+ // ~ only the very first comment
Review Comment:
i'll revise the comment; it's that only the first comment appearing right after the SELECT, INSERT... keywords can be a valid hint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "xitep (via GitHub)" <gi...@apache.org>.
xitep commented on code in PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162#discussion_r2727049681
##########
src/parser/mod.rs:
##########
@@ -14005,6 +14015,59 @@ impl<'a> Parser<'a> {
})
}
+ /// Parses an optional optimizer hint at the current token position
+ ///
+ /// [MySQL](https://dev.mysql.com/doc/refman/8.4/en/optimizer-hints.html#optimizer-hints-overview)
+ /// [Oracle](https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/Comments.html#GUID-D316D545-89E2-4D54-977F-FC97815CD62E)
+ fn parse_optional_optimizer_hint(&mut self) -> Result<Option<OptimizerHint>, ParserError> {
+ let supports_multiline = dialect_of!(self is MySqlDialect | OracleDialect | GenericDialect);
+ let supports_singleline = dialect_of!(self is OracleDialect | GenericDialect);
Review Comment:
i'm not sure `-- + foo` is a valid hint for mysql (i believe it's not.) that's why i did the separation in the first place. on oracle a `--+ quux` as valid as `/*+ quux */`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [MySQL, Oracle] Parse optimizer hints [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio merged PR #2162:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org