You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Taragolis (via GitHub)" <gi...@apache.org> on 2023/03/01 10:44:23 UTC

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

Taragolis commented on code in PR #29761:
URL: https://github.com/apache/airflow/pull/29761#discussion_r1121505755


##########
airflow/providers/amazon/aws/operators/ecs.py:
##########
@@ -93,31 +97,44 @@ def __init__(
         cluster_name: str,
         create_cluster_kwargs: dict | None = None,
         wait_for_completion: bool = True,
+        waiter_delay: int | None = None,
+        waiter_max_attempts: int | None = None,
         **kwargs,
     ) -> None:
         super().__init__(**kwargs)
         self.cluster_name = cluster_name
         self.create_cluster_kwargs = create_cluster_kwargs or {}
         self.wait_for_completion = wait_for_completion
+        self.waiter_delay = waiter_delay
+        self.waiter_max_attempts = waiter_max_attempts
 
     def execute(self, context: Context):
         self.log.info(
-            "Creating cluster %s using the following values: %s",
+            "Creating cluster %r using the following values: %s",
             self.cluster_name,
             self.create_cluster_kwargs,
         )
         result = self.client.create_cluster(clusterName=self.cluster_name, **self.create_cluster_kwargs)
-
-        if self.wait_for_completion:
-            while not EcsClusterStateSensor(
-                task_id="await_cluster",
-                cluster_name=self.cluster_name,
-            ).poke(context):
-                # The sensor has a built-in delay and will try again until
-                # the cluster is ready or has reached a failed state.
-                pass
-
-        return result["cluster"]
+        cluster_details = result["cluster"]
+        cluster_state = cluster_details.get("status")
+
+        if cluster_state == EcsClusterStates.ACTIVE:

Review Comment:
   Aaaaand this another Ctrl+C, Ctrl+V issue. The comment is a "lier", I will change it because it should stand for "is **created** immediately". With default provisioners ECS cluster is created during API call [CreateCluster](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCluster.html) just a personal findings based on years of usage ECS



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org