You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "sarkch (via GitHub)" <gi...@apache.org> on 2024/08/31 18:47:53 UTC
[I] StepFunctionStartExecutionOperator in MWAA does not throw error if IAM Role does not DescribeExecution permission [airflow]
sarkch opened a new issue, #41918:
URL: https://github.com/apache/airflow/issues/41918
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.8.1
### What happened?
I am using StepFunctionStartExecutionOperator to execute a StateMachine.
```
StepFunctionStartExecutionOperator(
...
task_id="load_data",
deferrable=True,
waiter_delay=30, # Poll for every 30 seconds
waiter_max_attempts=10, # maximum number of attempts to poll for status
do_xcom_push=True,
)
```
if I look into the log of the task,
{{waiter_with_logging.py:129}} INFO - Status of step function execution is:
{{waiter_with_logging.py:129}} INFO - Status of step function execution is:
{{waiter_with_logging.py:129}} INFO - Status of step function execution is:
as you can see that airflow is not getting the current status (RUNNING, FAILED etc) of the StateMachine
### What you think should happen instead?
Expected Output when the State Machine is RUNNING
```
[2024-08-28, 17:01:06 UTC] {{waiter_with_logging.py:129}} INFO - Status of step function execution is: RUNNING
[2024-08-28, 17:02:06 UTC] {{waiter_with_logging.py:129}} INFO - Status of step function execution is: RUNNING
```
### How to reproduce
So in my case I found the root cause of this problem. The IAM Role associated with the Airflow did not have the below permission
```
'states:DescribeExecution'
```
On the StateMachine execution arn.
```
arn:aws:states:<Region>:<accountId>:execution:<stateMachineName>:*
```
## Before granting the permission
```
[2024-08-28, 01:41:53 UTC] {{waiter_with_logging.py:129}} INFO - Status of step function execution is:
```
### After Granting the permission
```
[2024-08-28, 17:01:06 UTC] {{waiter_with_logging.py:129}} INFO - Status of step function execution is: RUNNING
```
### Operating System
Managed Airflow
### Versions of Apache Airflow Providers
_No response_
### Deployment
Amazon (AWS) MWAA
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] StepFunctionStartExecutionOperator in MWAA does not throw error if IAM Role does not DescribeExecution permission [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on issue #41918:
URL: https://github.com/apache/airflow/issues/41918#issuecomment-2323434363
Thanks for opening this, Looks like valid issue, able to re produce, these kind of errors validation is missing in async_wait.
Will push the changes for this.
cc: @eladkal
<img width="775" alt="image" src="https://github.com/user-attachments/assets/1581ffca-fb61-414b-aaa9-ed15cf72b118">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] StepFunctionStartExecutionOperator in MWAA does not throw error if IAM Role does not DescribeExecution permission [airflow]
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck closed issue #41918: StepFunctionStartExecutionOperator in MWAA does not throw error if IAM Role does not DescribeExecution permission
URL: https://github.com/apache/airflow/issues/41918
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org