You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "yoavcloud (via GitHub)" <gi...@apache.org> on 2025/04/10 09:18:48 UTC
[PR] Snowflake COPY INTO target columns, regular selec items and optional alias [datafusion-sqlparser-rs]
yoavcloud opened a new pull request, #1805:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1805
This PR adds support for the following in Snowflake's COPY INTO:
1. Specifying a list of columns in the destination table:
```sql
COPY INTO [<namespace>.]<table_name> [ ( <col_name> [ , <col_name> ... ] ) ]
```
2. Using regular SQL expressions in the data transformation query, typically for constant or computed values:
```sql
SELECT t1.$1:st AS st, $1:index, t2.$1, 4, '5' AS const_str FROM @schema.general_finished
```
3. The `AS` keyword is optional when specifying an alias for the stage in the data transformation query
```sql
COPY INTO tbl FROM (SELECT t1.$1:st AS st, 4, '5' AS const_str FROM @stage S)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Snowflake COPY INTO target columns, select items and optional alias [datafusion-sqlparser-rs]
Posted by "yoavcloud (via GitHub)" <gi...@apache.org>.
yoavcloud commented on code in PR #1805:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1805#discussion_r2039169752
##########
src/dialect/snowflake.rs:
##########
@@ -866,86 +867,97 @@ pub fn parse_copy_into(parser: &mut Parser) -> Result<Statement, ParserError> {
fn parse_select_items_for_data_load(
parser: &mut Parser,
-) -> Result<Option<Vec<StageLoadSelectItem>>, ParserError> {
- // [<alias>.]$<file_col_num>[.<element>] [ , [<alias>.]$<file_col_num>[.<element>] ... ]
- let mut select_items: Vec<StageLoadSelectItem> = vec![];
+) -> Result<Option<Vec<StageLoadSelectItemKind>>, ParserError> {
+ let mut select_items: Vec<StageLoadSelectItemKind> = vec![];
loop {
- let mut alias: Option<Ident> = None;
- let mut file_col_num: i32 = 0;
- let mut element: Option<Ident> = None;
- let mut item_as: Option<Ident> = None;
+ match parser.maybe_parse(parse_select_item_for_data_load)? {
+ // [<alias>.]$<file_col_num>[.<element>] [ , [<alias>.]$<file_col_num>[.<element>] ... ]
+ Some(item) => select_items.push(StageLoadSelectItemKind::StageLoadSelectItem(item)),
+ // Fallback, try to parse a standard SQL select item
+ None => select_items.push(StageLoadSelectItemKind::SelectItem(
+ parser.parse_select_item()?,
+ )),
+ }
+ match parser.next_token().token {
+ Token::Comma => {
+ // continue
+ }
+ _ => {
+ parser.prev_token(); // need to move back
+ break;
+ }
+ }
Review Comment:
Great idea! I included it in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Snowflake COPY INTO target columns, regular selec items and optional alias [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio commented on code in PR #1805:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1805#discussion_r2038845330
##########
src/dialect/snowflake.rs:
##########
@@ -866,86 +867,97 @@ pub fn parse_copy_into(parser: &mut Parser) -> Result<Statement, ParserError> {
fn parse_select_items_for_data_load(
parser: &mut Parser,
-) -> Result<Option<Vec<StageLoadSelectItem>>, ParserError> {
- // [<alias>.]$<file_col_num>[.<element>] [ , [<alias>.]$<file_col_num>[.<element>] ... ]
- let mut select_items: Vec<StageLoadSelectItem> = vec![];
+) -> Result<Option<Vec<StageLoadSelectItemKind>>, ParserError> {
+ let mut select_items: Vec<StageLoadSelectItemKind> = vec![];
loop {
- let mut alias: Option<Ident> = None;
- let mut file_col_num: i32 = 0;
- let mut element: Option<Ident> = None;
- let mut item_as: Option<Ident> = None;
+ match parser.maybe_parse(parse_select_item_for_data_load)? {
+ // [<alias>.]$<file_col_num>[.<element>] [ , [<alias>.]$<file_col_num>[.<element>] ... ]
+ Some(item) => select_items.push(StageLoadSelectItemKind::StageLoadSelectItem(item)),
+ // Fallback, try to parse a standard SQL select item
+ None => select_items.push(StageLoadSelectItemKind::SelectItem(
+ parser.parse_select_item()?,
+ )),
+ }
+ match parser.next_token().token {
+ Token::Comma => {
+ // continue
+ }
+ _ => {
+ parser.prev_token(); // need to move back
+ break;
+ }
+ }
Review Comment:
could we do something like this in order to avoid the token clone here?
```rust
if matches!(parser.peek_token_ref(), Token::Comma) {
parser.advance()
} else {
break
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Snowflake COPY INTO target columns, select items and optional alias [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio merged PR #1805:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org