You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/26 15:45:32 UTC

[GitHub] [airflow] gfelot opened a new issue #10585: googleCloudStorageDeleteOperator URL encode "/"

gfelot opened a new issue #10585:
URL: https://github.com/apache/airflow/issues/10585


   **Apache Airflow version**: 1.10.9
   
   **What happened**:
   
   I want to delete temporary file at the end of my DAG in a GCS bucket. When I add the list of values in the arguments `object` I get an error that the file is not found. The reason is the symbole **/** is URL encoded **%2F**.
   
   ```
   gcs_delete_temp = GoogleCloudStorageDeleteOperator(
       task_id="gcs_delete_temp",
       bucket_name=f"super_bucket",
       objects=[
           "rte/temp/{{ macros.ds_add(ds, -10) }}_injectee.parquet",
           "rte/temp/{{ macros.ds_add(ds, -10) }}_soutiree.parquet"
       ]
   )
   ```
   I got the error:  
   
   > google.api_core.exceptions.NotFound: 404 DELETE https://storage.googleapis.com/storage/v1/b/super_bucket/o/rte%2Ftemp%2F2020-07-25_injectee.parquet: Not Found
   
   I tried to escape them but the **\** symbole is URL encoded too.
   
   Is that a bug ? Or a feature ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] duthades commented on issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
duthades commented on issue #10585:
URL: https://github.com/apache/airflow/issues/10585#issuecomment-721209004


   Hi, I would like to attempt to solve this issue. Can you assign me?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #10585:
URL: https://github.com/apache/airflow/issues/10585#issuecomment-724041936


   So closing the issue then


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #10585:
URL: https://github.com/apache/airflow/issues/10585


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] duthades commented on issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
duthades commented on issue #10585:
URL: https://github.com/apache/airflow/issues/10585#issuecomment-721324661


   This behaviour is correct. ```\``` is encoded when it appears in either the object name or query string of a request URI. Other characters which are encoded can be found [here](https://cloud.google.com/storage/docs/request-endpoints#encoding).
   
   As far as the error ```404``` is concerned, it is because either [bucket or object name was not found](https://cloud.google.com/storage/docs/xml-api/reference-status#404%E2%80%94not-found).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] duthades removed a comment on issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
duthades removed a comment on issue #10585:
URL: https://github.com/apache/airflow/issues/10585#issuecomment-721209004


   Hi, I would like to attempt to solve this issue. Can you assign me?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] duthades commented on issue #10585: googleCloudStorageDeleteOperator URL encode "/"

Posted by GitBox <gi...@apache.org>.
duthades commented on issue #10585:
URL: https://github.com/apache/airflow/issues/10585#issuecomment-724035417


   @eladkal I think the ```kind:bug``` tag can be removed as it's not a bug.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org