You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/29 10:23:54 UTC

[GitHub] [airflow] druthven opened a new issue, #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

druthven opened a new issue, #25395:
URL: https://github.com/apache/airflow/issues/25395

   ### Description
   
   Require an operator similar to the S3ToSnowflakeOperator but for GCS to load data stored in GCS to a Snowflake table.
   
   ### Use case/motivation
   
   Same as the S3ToSnowflakeOperator.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1199165879

   Marked it as good first issue - maybe someone will pick it up- but FYI @druthven - in OSS project like this, such features like this get implemented when .... someone implements it. So the best way (if you are really into getting this to happne) is to implement it yourself and become one of the many contributors (> 2100)  - most of them users like you who had the need for something and implemented it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1199115803

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] Taragolis commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1200128020

   @eladkal I could start but at the moment I have problems with time balance. So if I start with this PR I am afraid it wouldn't happen before mid of August.
   
   If someone want work on this feature right now I will be happy to give the opportunity to extend the existing operator to someone else.
   
   In case if it won't implements at moment when I ready to work on this PR I will create PR definitely. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] blcksrx commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
blcksrx commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1200258580

   I can work on it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] Taragolis commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
Taragolis commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1199958695

   @potiuk @blcksrx Just want to mention that right now S3ToSnowflakeOperator it is just wrapper around [COPY INTO \<table>](https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html#syntax) where location is `externalStage`.  Due to the fact that operator only uses internal snowflake capabilities: [STAGE](https://docs.snowflake.com/en/sql-reference/sql/create-stage.html) it should work not only with AWS S3 but also with GCS and Azure Containers.
   
   Might be better rename(deprecate)`S3ToSnowflakeOperator` to something like `CopyIntoTableFromExternalStageOperator`, change `s3_keys` to `files` and mention that it works with AWS S3, GCS and Azure Containers
   
   cc: @druthven 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1200096683

   @Taragolis can you start a PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] blcksrx commented on issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
blcksrx commented on issue #25395:
URL: https://github.com/apache/airflow/issues/25395#issuecomment-1199944621

   I would like to do it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #25395: GCSToSnowflakeOperator with feature parity to the S3ToSnowflakeOperator
URL: https://github.com/apache/airflow/issues/25395


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org