[GitHub] [airflow] Yao-ATG commented on issue #22675: GCSToGCSOperator cannot copy a single file/folder without copying other files/folders with that prefix

[GitBox <gi...@apache.org>]

Yao-ATG commented on issue #22675:
URL: https://github.com/apache/airflow/issues/22675#issuecomment-1086027371


   > See the documentation (docstring) where you can have examples.
   > 
   > As unitutitive as it is, source_object is a wildcard specification by default. If you want to copy single object you need specify it like that:
   > 
   > ```
   > source_objects = [ 'your_object' ] 
   > ```
   > 
   > See examples here: https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_api/airflow/providers/google/cloud/transfers/gcs_to_gcs/index.html
   
   Unfortunately using source_objects instead of source_object doesn't help.
   I verified it, by running the DAG,  both before opening the issue and after your reply.
   Also from source code we can see there is no difference between specifying an file in source_object and in source_objects.
   https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/gcs_to_gcs.py, line 246 we have
           if self.source_object:
               self.source_objects = [self.source_object]
   and we work only on source_objects afterwords, treating the object as prefix.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org