You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/14 18:40:49 UTC

[GitHub] [airflow] dstandish commented on pull request #12505: Fix S3ToSnowflakeOperator to support uploading all files in the specified stage

dstandish commented on pull request #12505:
URL: https://github.com/apache/airflow/pull/12505#issuecomment-760392600


   one observation, which we might not want to deal with at this time but i'll make it anyway..
   
   snowflake supports azure  / s3 / gcs
   
   this operator assumes a stage and not any particular storage backend, so i am pretty sure it works identically with gcp and azure.
   
   so it might be we should deprecate this and rename it not as a transfer operator but a SnowflakeCopyOperator
   
   additionally, i think it should eventually be updated to parse the results and fail as appropriate, which could depend on whether there are skipped or failed files and should be configurable.  but i'm sure you'll want to consider this out of scope on this change.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org