You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "barsela1 (via GitHub)" <gi...@apache.org> on 2025/04/06 11:58:27 UTC
[PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
barsela1 opened a new pull request, #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Snowflake: support nested join without parentheses [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio merged PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2036518287
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
Ah yeah so what I meant was that the current `parse_table_factor` gets renamed to something like `parse_table_factor_inner`, then the actual impl `parse_table_factor`, after parsing a lone `TableFactor` here
```rust
let table_factor = self.parse_table_factor_inner()?;
```
it next peeks if this table factor is actually a nested join, similar to the current PR impl, and if it is, completes the parsing as a nested table factor
```rust
let mut table_factor = self.parse_table_factor_inner()?;
if self.peek_parens_less_nested_join() {
let joins = self.parse_joins()?;
table_factor = TableFactor::NestedJoin {
table_with_joins: Box::new(TableWithJoins { relation: table_factor, joins }),
alias: None,
};
}
Ok(table_factor)
```
would this be reasonable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2043675390
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
Ah I see, that sounds reasonable then thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "barsela1 (via GitHub)" <gi...@apache.org>.
barsela1 commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2034746065
##########
tests/sqlparser_snowflake.rs:
##########
@@ -3516,3 +3516,87 @@ fn test_alter_session_followed_by_statement() {
_ => panic!("Unexpected statements: {:?}", stmts),
}
}
+
+#[test]
+fn test_nested_join_without_parentheses() {
+ let query = "SELECT DISTINCT p.product_id FROM orders AS o INNER JOIN customers AS c INNER JOIN products AS p ON p.customer_id = c.customer_id ON c.order_id = o.order_id";
+ assert_eq!(
+ only(
+ snowflake()
+ .verified_only_select_with_canonical(query, "")
Review Comment:
I want to skip the canonical comparison because parentheses were added there and it fails the test even though it is correct. Is there a better way to overcome this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "barsela1 (via GitHub)" <gi...@apache.org>.
barsela1 commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2035040782
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
Can you please explain the example you brought?
if i do so, the test "test_join" fail...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2035155339
##########
tests/sqlparser_snowflake.rs:
##########
@@ -3516,3 +3516,87 @@ fn test_alter_session_followed_by_statement() {
_ => panic!("Unexpected statements: {:?}", stmts),
}
}
+
+#[test]
+fn test_nested_join_without_parentheses() {
+ let query = "SELECT DISTINCT p.product_id FROM orders AS o INNER JOIN customers AS c INNER JOIN products AS p ON p.customer_id = c.customer_id ON c.order_id = o.order_id";
+ assert_eq!(
+ only(
+ snowflake()
+ .verified_only_select_with_canonical(query, "")
Review Comment:
ah I see, yeah in that case I think we can include the canonical representation to assert on in place of the empty string. [Here for example](https://github.com/apache/datafusion-sqlparser-rs/blob/c788526151d958a60896fcef156306af8ba99ea5/tests/sqlparser_snowflake.rs#L2717)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "barsela1 (via GitHub)" <gi...@apache.org>.
barsela1 commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2036691068
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
I’m not sure this approach is correct...test_join() test fails.
In parse_table_and_joins, there's an initial call to parse_table_factor that is unrelated to join parsing.
With this implementation, the first table will always be parsed as a NestedJoin, even when it shouldn’t be.
I tried to handle nested joins explicitly within the join logic to avoid that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2034488630
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
would it make more sense to have this logic as part of the table factor parsing instead? thinking that would be by definition, and so that any caller of `self.parse_table_factor` gets the full behavior including nested joins.
i imagine essentially to split table factor parsing instead e.g.
```rust
fn parse_table_factor() {
let table_factor = self.parse_table_factor_inner()?;
if self.is_next_token_a_join() {
// ...
}
}
```
##########
tests/sqlparser_snowflake.rs:
##########
@@ -3516,3 +3516,87 @@ fn test_alter_session_followed_by_statement() {
_ => panic!("Unexpected statements: {:?}", stmts),
}
}
+
+#[test]
+fn test_nested_join_without_parentheses() {
+ let query = "SELECT DISTINCT p.product_id FROM orders AS o INNER JOIN customers AS c INNER JOIN products AS p ON p.customer_id = c.customer_id ON c.order_id = o.order_id";
Review Comment:
Can we add coverage for all the supported join types?
##########
tests/sqlparser_snowflake.rs:
##########
@@ -3516,3 +3516,87 @@ fn test_alter_session_followed_by_statement() {
_ => panic!("Unexpected statements: {:?}", stmts),
}
}
+
+#[test]
+fn test_nested_join_without_parentheses() {
+ let query = "SELECT DISTINCT p.product_id FROM orders AS o INNER JOIN customers AS c INNER JOIN products AS p ON p.customer_id = c.customer_id ON c.order_id = o.order_id";
+ assert_eq!(
+ only(
+ snowflake()
+ .verified_only_select_with_canonical(query, "")
Review Comment:
oh what does the empty string second argument imply?
##########
src/parser/mod.rs:
##########
@@ -11833,7 +11847,21 @@ impl<'a> Parser<'a> {
};
joins.push(join);
}
- Ok(TableWithJoins { relation, joins })
+ Ok(joins)
+ }
+
+ fn is_next_token_a_join(&self) -> bool {
+ matches!(
+ self.peek_token().token,
Review Comment:
```suggestion
self.peek_token_ref().token,
```
##########
src/parser/mod.rs:
##########
@@ -11833,7 +11847,21 @@ impl<'a> Parser<'a> {
};
joins.push(join);
}
- Ok(TableWithJoins { relation, joins })
+ Ok(joins)
+ }
+
+ fn is_next_token_a_join(&self) -> bool {
Review Comment:
```suggestion
fn peek_parens_less_nested_join(&self) -> bool {
```
maybe something like this naming wise? peek uses existing parser terminology. Also would it be possible with a doc comment of the function mentioning the use case? I think that would be helpful to future readers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "barsela1 (via GitHub)" <gi...@apache.org>.
barsela1 commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2035040782
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
Can you please explain the example you brought? i didn't fully understood it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] add support to nested join_without parentheses snowflake [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1799:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1799#discussion_r2036518287
##########
src/parser/mod.rs:
##########
@@ -11823,7 +11828,16 @@ impl<'a> Parser<'a> {
}
_ => break,
};
- let relation = self.parse_table_factor()?;
+ let mut relation = self.parse_table_factor()?;
+
+ if self.is_next_token_a_join() {
+ let joins = self.parse_joins()?;
+ relation = TableFactor::NestedJoin {
+ table_with_joins: Box::new(TableWithJoins { relation, joins }),
+ alias: None,
+ };
+ }
Review Comment:
Ah yeah so what I meant was that the current `parse_table_factor` gets renamed to something like `parse_table_factor_inner`, then the new impl `parse_table_factor` becomes a wrapper around the inner, such that after parsing a lone `TableFactor` here
```rust
let table_factor = self.parse_table_factor_inner()?;
```
it next peeks if this table factor is actually a nested join, similar to the current PR impl, and if it is, completes the parsing as a nested table factor
```rust
let mut table_factor = self.parse_table_factor_inner()?;
if self.peek_parens_less_nested_join() {
let joins = self.parse_joins()?;
table_factor = TableFactor::NestedJoin {
table_with_joins: Box::new(TableWithJoins { relation: table_factor, joins }),
alias: None,
};
}
Ok(table_factor)
```
would this be reasonable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org