You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/29 03:01:33 UTC

[GitHub] [airflow] RNHTTR opened a new issue #13966: Create a Google Drive Sensor and a Google Drive to Google Cloud Storage Operator

RNHTTR opened a new issue #13966:
URL: https://github.com/apache/airflow/issues/13966


   <!--
   
   Welcome to Apache Airflow!  For a smooth issue process, try to answer the following questions.
   Don't worry if they're not all applicable; just try to include what you can :-)
   
   If you need to include code snippets or logs, please put them in fenced code
   blocks.  If they're super-long, please use the details tag like
   <details><summary>super-long log</summary> lots of stuff </details>
   
   Please delete these comment blocks before submitting the issue.
   
   -->
   
   **Description**
   
   <!-- A short description of your feature -->
   
   - Google Drive Sensor `airflow/providers/google/suite/sensors/drive.py`
     - This sensor would check for the existence of a file in Google Drive.
   - Google Drive to Google Cloud Storage Operator `airflow/providers/google/cloud/transfers/gdrive_to_gcs.py`
     - This transfer operator would copy files from a shared Google Drive folder to a location in a Google Cloud Storage bucket.
   - Additional functionality to the Google Drive Hook `airflow/providers/google/suite/hooks/drive.py` to support the operator & sensor.
   
   **Use case / motivation**
   - ETL
     - The transfer operator can be used by non-technical users trigger ETL pipelines. For example, a user can upload a CSV file to a shared Google Drive folder to be ingested into BigQuery. The sensor would be used to first ensure the existence of a particular file in Google Drive before the operator attempts to copy it.
   - Backup / archival
     - The transfer operator can create backups or archives of Drive files in Google Cloud Storage.
   
   
   <!-- What do you want to happen?
   
   Rather than telling us how you might implement this solution, try to take a
   step back and describe what you are trying to achieve.
   
   -->
   
   **Are you willing to submit a PR?**
   Yes! The code has been developed and works as expected in Airflow/Breeze.
   <!--- We accept contributions! -->
   
   **Related Issues**
   N/a
   <!-- Is there currently another issue associated with this? -->
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #13966: Create a Google Drive Sensor and a Google Drive to Google Cloud Storage Operator

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13966:
URL: https://github.com/apache/airflow/issues/13966


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13966: Create a Google Drive Sensor and a Google Drive to Google Cloud Storage Operator

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13966:
URL: https://github.com/apache/airflow/issues/13966#issuecomment-769541672


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #13966: Create a Google Drive Sensor and a Google Drive to Google Cloud Storage Operator

Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #13966:
URL: https://github.com/apache/airflow/issues/13966#issuecomment-769642401


   > If you create pull-request, you don't have to create an issue first, but if you want, you can do it. Creating an issue will allow you to collect feedback or share plans with other people.
   
   https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow
   
   I am looking forward for your PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org