You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/27 21:27:26 UTC

[GitHub] [airflow] ephraimbuddy opened a new issue #9040: Correctly describe the dst parameter in LocalFilesystemToGCS operator

ephraimbuddy opened a new issue #9040:
URL: https://github.com/apache/airflow/issues/9040


   
   **Apache Airflow version**: 2.0.0
   
   
   **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
   
   **Environment**:
   
   - **Cloud provider or hardware configuration**:
   - **OS** (e.g. from /etc/os-release):
   - **Kernel** (e.g. `uname -a`):
   - **Install tools**:
   - **Others**:
   
   **What happened**:
   
   The description of the `dst` parameter of the LocalFilesystemToGCSOperator is not correct. 
   https://github.com/apache/airflow/blob/738667082d32d3ef93ec2cd6c3735ff3691ba1cc/airflow/providers/google/cloud/operators/local_to_gcs.py#L35-L36
   
   **What you expected to happen**:
   
   I expected it to be described as the object name. The `dst` parameter was simply passed to the `object_name` parameter of the `upload` method of the `GCSHook`. This means that the `dst` parameter is the object name to use to store the uploaded object and not a path as described. 
   https://github.com/apache/airflow/blob/738667082d32d3ef93ec2cd6c3735ff3691ba1cc/airflow/providers/google/cloud/operators/local_to_gcs.py#L90-L96
   Here is the description of the `object_name` parameter in the upload method of `GCSHook`:  `The object name to set when uploading the file.`
   https://github.com/apache/airflow/blob/738667082d32d3ef93ec2cd6c3735ff3691ba1cc/airflow/providers/google/cloud/hooks/gcs.py#L300-L323
   
   **How to reproduce it**:
   
   
   **Anything else we need to know**:
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #9040: Correctly describe the dst parameter in LocalFilesystemToGCS operator

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #9040:
URL: https://github.com/apache/airflow/issues/9040#issuecomment-635470492


   It's a bit ambiguous indeed and it would be great to make it consistent. Objects in GCS are often referred to as files - even if they are really objects - but in all kinds of UI  interfaces the objects with "/" are interpreted as folders/paths (but they are really not paths -  just object names). There is no real concept of "folder" in GCS.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] turbaszek closed issue #9040: Correctly describe the dst parameter in LocalFilesystemToGCS operator

Posted by GitBox <gi...@apache.org>.
turbaszek closed issue #9040:
URL: https://github.com/apache/airflow/issues/9040


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9040: Correctly describe the dst parameter in LocalFilesystemToGCS operator

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9040:
URL: https://github.com/apache/airflow/issues/9040#issuecomment-634950607


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org