You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/19 16:59:56 UTC
[GitHub] [airflow] Taragolis commented on a diff in pull request #28464: Add waiter config params to emr.add_job_flow_steps
Taragolis commented on code in PR #28464:
URL: https://github.com/apache/airflow/pull/28464#discussion_r1052434511
##########
airflow/providers/amazon/aws/hooks/emr.py:
##########
@@ -126,14 +126,21 @@ def create_job_flow(self, job_flow_overrides: dict[str, Any]) -> dict[str, Any]:
return response
def add_job_flow_steps(
- self, job_flow_id: str, steps: list[dict] | str | None = None, wait_for_completion: bool = False
+ self,
+ job_flow_id: str,
+ steps: list[dict] | str | None = None,
+ wait_for_completion: bool = False,
+ waiter_delay: int = 5,
+ waiter_max_attempts: int = 100,
Review Comment:
Hmm.. what do you think if by default `waiter_delay` and `waiter_max_attempts` set to `None` and use in waiter config only if this values not None?
In this case default values from `botocore` waiters definition file will use: delay=30, max_attempts=60.
Something like that
```python
waiter.wait(
ClusterId=job_flow_id,
StepId=step_id,
WaiterConfig=trim_none_values(
{
"Delay": waiter_delay,
"MaxAttempts": waiter_max_attempts,
}
),
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org