You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "match-jasoncavnar (via GitHub)" <gi...@apache.org> on 2023/10/20 20:06:54 UTC

[I] Conditional Task Skip option [airflow]

match-jasoncavnar opened a new issue, #35089:
URL: https://github.com/apache/airflow/issues/35089

   ### Description
   
   Add a keyword option at the task level to conditionally set if the task should be skipped on DAG run.
   
   ### Use case/motivation
   
   We would like to run our DAG in three environments; Dev, QA, and Prod. In Dev and QA we have sensors and file download tasks that would fail because the files do not exist in the environments. The skip keyword would allow us to conditionally set if the task should be skipped based on the DAG running in our DEV and QA envrionments.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Conditional Task Skip option [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #35089:
URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773331065

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Conditional Task Skip option [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #35089: Conditional Task Skip option
URL: https://github.com/apache/airflow/issues/35089


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Conditional Task Skip option [airflow]

Posted by "abhishekbhakat (via GitHub)" <gi...@apache.org>.
abhishekbhakat commented on issue #35089:
URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773678144

   Why don't you just set different environment variables for your environments and use AirflowSkipException to skip a task.
   Something like:
   ```python
           if os.environ.get("ENVIRONMENT") == "dev":
               raise AirflowSkipException("Skipping this task in dev")
   ```
   
   This doesn't need a feature.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org