[GitHub] [airflow] nuclearpinguin commented on issue #7585: [AIRFLOW-6944] Allow AWS DataSync to "catch up" when Task is already …

[GitBox <gi...@apache.org>]

nuclearpinguin commented on issue #7585: [AIRFLOW-6944] Allow AWS DataSync to "catch up" when Task is already …
URL: https://github.com/apache/airflow/pull/7585#issuecomment-599725428
 
 
   > 1. Wait for task to be able to be started - implemented in the DataSync Operator.
   > 2. Wait for task execution to be completed - implemented in the DataSync hook. (Task execution is what you get from AWS after triggering a Task to start)
   > 
   > What is your recommendation to standardise these?
   
   I would say that we should not wait for task to be able to start. If something has to happen before the task we should use sensor or wait for the completion in the upstream operator. For example in case of `CreateResource >> UseResource` the `CreateResource` op should check if the resource was successfully created. This is the approach we use in GCP operators/hooks. 
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services