You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/06/20 22:35:53 UTC
[GitHub] [airflow] Taragolis opened a new pull request, #24571: Pattern parameter in S3ToSnowflakeOperator
Taragolis opened a new pull request, #24571:
URL: https://github.com/apache/airflow/pull/24571
closes: #24360
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr merged pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
uranusjr merged PR #24571:
URL: https://github.com/apache/airflow/pull/24571
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161978990
Test on real Snowflake COPY INTO instruction all `'` in `PATTERN` should be replaced by two single quotes as well as in `FILES`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161318469
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161498424
> `files` is also currently not escaped or validated,
Just check and found that files actually also quoted however not validated.
https://github.com/apache/airflow/blob/904cc121b83ecfaacba25433a7911a2541b2c312/airflow/providers/snowflake/transfers/s3_to_snowflake.py#L122
Single quote is a [safe to use special character](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines-safe-characters) for S3 key/prefixes.
That mean if user has `'` in S3 key in current implementation most probably it would fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
josh-fell commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161804201
CC @grzegorzb1990 since the issue was assigned to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161323814
Doc build failure is unrelated (see #24575).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on pull request #24571: Pattern parameter in S3ToSnowflakeOperator
Posted by GitBox <gi...@apache.org>.
Taragolis commented on PR #24571:
URL: https://github.com/apache/airflow/pull/24571#issuecomment-1161363353
> No expertise to judge whether `pattern` needs special escaping (e.g. what if the pattern contains a `'`?) but `files` is also currently not escaped or validated, so perhaps not? And if there’s a bug, we need to fix them all anyway, and that should happen in a separate PR, not this one.
This is real good question. About `pattern` it should always be escaped by single quote ([doc](https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#optional-parameters)) but It unclear what if pattern itself should contain single quote. I agree that at least it should be mention in documentation/docstring.
Never use option `files` for Snowflake, seems like it should also escaped, but documentation unclear about this.
Anyway I will try to check on actual Snowflake during the day different cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org