You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "match-jasoncavnar (via GitHub)" <gi...@apache.org> on 2023/10/20 20:06:54 UTC
[I] Conditional Task Skip option [airflow]
match-jasoncavnar opened a new issue, #35089:
URL: https://github.com/apache/airflow/issues/35089
### Description
Add a keyword option at the task level to conditionally set if the task should be skipped on DAG run.
### Use case/motivation
We would like to run our DAG in three environments; Dev, QA, and Prod. In Dev and QA we have sensors and file download tasks that would fail because the files do not exist in the environments. The skip keyword would allow us to conditionally set if the task should be skipped based on the DAG running in our DEV and QA envrionments.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Conditional Task Skip option [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #35089:
URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773331065
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Conditional Task Skip option [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #35089: Conditional Task Skip option
URL: https://github.com/apache/airflow/issues/35089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Conditional Task Skip option [airflow]
Posted by "abhishekbhakat (via GitHub)" <gi...@apache.org>.
abhishekbhakat commented on issue #35089:
URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773678144
Why don't you just set different environment variables for your environments and use AirflowSkipException to skip a task.
Something like:
```python
if os.environ.get("ENVIRONMENT") == "dev":
raise AirflowSkipException("Skipping this task in dev")
```
This doesn't need a feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org