[GitHub] [airflow] josh-fell commented on a change in pull request #19665: Add two operators in AWS Providers: RedshiftResumeClusterOperator and RedshiftPauseClusterOperator

[GitBox <gi...@apache.org>]

josh-fell commented on a change in pull request #19665:
URL: https://github.com/apache/airflow/pull/19665#discussion_r753948606



##########
File path: airflow/providers/amazon/aws/hooks/redshift.py
##########
@@ -138,6 +138,26 @@ def create_cluster_snapshot(self, snapshot_identifier: str, cluster_identifier:
         )
         return response['Snapshot'] if response['Snapshot'] else None
 
+    def wait_for_state(self, cluster_identifier: str, target_state: str, check_interval: float) -> None:

Review comment:
       There is a design pattern in several operators that include the option, rather than a forced action, to wait for termination/completion if desired. 
   https://github.com/apache/airflow/blob/main/airflow/providers/microsoft/azure/operators/data_factory.py
   https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/operators/dataflow.py
   https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/operators/dataproc.py
   https://github.com/apache/airflow/blob/main/airflow/providers/tableau/operators/tableau_refresh_workbook.py
   
   This way users could choose whether to wait for the task to complete as part of the operator or use a sensor to poke for status (mainly useful for long-running actions). Of course, Deferrable Operators solve for this too 🙂 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org