You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/22 18:02:30 UTC

[GitHub] [airflow] denimalpaca opened a new issue #17168: Add S3Load Operator to transfer local files to an S3 location

denimalpaca opened a new issue #17168:
URL: https://github.com/apache/airflow/issues/17168


   **Description**
   
   Currently, an S3Hook exists that allows transfer of files to S3 via `load_file()`, however there is no operator associated with it. The S3Load Operator would wrap the S3 Hook, so it is not used directly.
   
   **Use case / motivation**
   
   Seeing as to upload a local file to S3 using the S3 Hook, a Python task with the same functionality has to be written anyway, this could reduce a lot of redundant boiler-plate code and standardize the local file to S3 load process.
   
   **Are you willing to submit a PR?**
   
   Yes
   
   **Related Issues**
   
   Not that I could find.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17168: Add S3Load Operator to transfer local files to an S3 location

Posted by GitBox <gi...@apache.org>.

boring-cyborg[bot] commented on issue #17168:
URL: https://github.com/apache/airflow/issues/17168#issuecomment-885122321


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #17168: Add LocalFilesystemtoS3Operator

Posted by GitBox <gi...@apache.org>.

potiuk closed issue #17168:
URL: https://github.com/apache/airflow/issues/17168


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #17168: Add S3Load Operator to transfer local files to an S3 location

Posted by GitBox <gi...@apache.org>.

eladkal commented on issue #17168:
URL: https://github.com/apache/airflow/issues/17168#issuecomment-886162138


   Yeah we can have an operator to load from local to S3 (like the equivalent from GCP ) https://github.com/apache/airflow/blob/45a49e0f81cc592a4e6a95aa230560530d88a5c8/airflow/providers/google/cloud/transfers/local_to_gcs.py#L28
   
   assigned the issue to you :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org