Re: [D] [proposal] provide DbtCloudRunJobOperator with an option re-poking the api endpoint if it's a http error [airflow]

["keerthi-reddiwandla (via GitHub)" <gi...@apache.org>]

GitHub user keerthi-reddiwandla edited a discussion: [proposal] provide DbtCloudRunJobOperator with an option re-poking the api endpoint if it's a http error

We host a dbt jobs DAG in MWAA using DbtCloudRunJobOperator. Sometimes the DbtCloudRunJobOperator get 404 error from the dbt cloud api endpoint even though it has triggered the dbt cloud job run.The task is marked as FAILED and it moves on to next task. It would be helpful to add options after providing following arguments: 
1.retry_attempts: if the error is http error, you can provide certain number of retry attempts.
2.retry_after_seconds: Initiate the retry after certain number of seconds.

I can look into opening a PR if you can guide me in a general direction of making this change.

GitHub link: https://github.com/apache/airflow/discussions/49424

----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org