Re: [PR] Fix the gcp_gcs_delete_objects operator delete whole bucket on processing empty list of objects [airflow]

["RemyOstyn (via GitHub)" <gi...@apache.org>]

RemyOstyn commented on PR #32383:
URL: https://github.com/apache/airflow/pull/32383#issuecomment-1842065967

   Hello @yupbank,
   
   I found the same issue when using **Airflow Task Group**. If we forget to specify the when pulling files paths from xcoms, the `objects` will be an empty array between double quotes `"[]"` and the **GCSDeleteObjectsOperator** will delete the whole bucket content.
   
   **Example:** (let's assume these tasks are in a task group)
   
   ```
   list_files_from_gcs = GCSListObjectsOperator(
               task_id="list_files_from_gcs",
               bucket="my-bucket",
               match_glob=path + "/*.csv",
           )
   
   remove_files_from_gcs = GCSDeleteObjectsOperator(
               task_id="remove_files_from_gcs",
               bucket_name="my-bucket,
               objects="{{ ti.xcom_pull('list_files_from_gcs') }}",
           )
   ```
   
   Instead of:
   
   ```
   list_files_from_gcs = GCSListObjectsOperator(
               task_id="list_files_from_gcs",
               bucket="my-bucket",
               match_glob=path + "/*.csv",
           )
   
   remove_files_from_gcs = GCSDeleteObjectsOperator(
               task_id="remove_files_from_gcs",
               bucket_name="my-bucket",
               objects="{{ ti.xcom_pull('my_task_group.list_files_from_gcs') }}", // As you can see I have to specify the task group here to avoid deleting the whole bucket
           )
   ```
   
   Is there a way to add protection when using task group?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org