You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/05 20:37:52 UTC

[GitHub] [airflow] timhealz opened a new issue #8724: Uploading List of Files to Google Cloud Storage

timhealz opened a new issue #8724:
URL: https://github.com/apache/airflow/issues/8724


   **Description**
   
   In the [current implementation](https://airflow.apache.org/docs/stable/_api/airflow/contrib/operators/file_to_gcs/index.html) of `FileToGoogleCloudStorageOperator`, a single file is inputted to be uploaded to Google Cloud Storage bucket.
   
   I'd like to implement functionality to allow input of a list of files, directory path or perhaps even a wildcard string (`/path/to/*.csv` for example).
   
   **Related Issues**
   
   Not that I can tell from searching, but there is this [stackoverflow question](https://stackoverflow.com/questions/58945050/uploading-local-directory-to-gcs-using-airflow) that @kaxil answered. 
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] timhealz commented on issue #8724: Uploading List of Files to Google Cloud Storage

Posted by GitBox <gi...@apache.org>.
timhealz commented on issue #8724:
URL: https://github.com/apache/airflow/issues/8724#issuecomment-624311670


   Agreed @mik-laj , I can work on the PR if this issue can be assigned to me.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] mik-laj commented on issue #8724: Uploading List of Files to Google Cloud Storage

Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8724:
URL: https://github.com/apache/airflow/issues/8724#issuecomment-624295975


   Very useful. I think it can easily be done if you use [glob.glob](https://docs.python.org/3/library/glob.html#glob.glob)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #8724: Uploading List of Files to Google Cloud Storage

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8724:
URL: https://github.com/apache/airflow/issues/8724#issuecomment-624316698


   > Agreed @mik-laj , I can work on the PR if this issue can be assigned to me.
   
   Done


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] turbaszek closed issue #8724: Uploading List of Files to Google Cloud Storage

Posted by GitBox <gi...@apache.org>.
turbaszek closed issue #8724:
URL: https://github.com/apache/airflow/issues/8724


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8724: Uploading List of Files to Google Cloud Storage

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8724:
URL: https://github.com/apache/airflow/issues/8724#issuecomment-624292915


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org