You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/11 19:49:17 UTC

[GitHub] [airflow] isaac-florence opened a new issue, #28293: Skipping dag runs on established daily DAG

isaac-florence opened a new issue, #28293:
URL: https://github.com/apache/airflow/issues/28293

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   Airflow 2.4.3, running on Kubernetes with High-Availability Scheduler (2 pods). Database is postgres 10.21
   
   I have a dag that has been running for two months with this timetable: `timetable=CronTriggerTimetable("40 5 * * *", timezone = 'UTC')` (which has not changed). It has run reliably every day as expected however yesterday it simply did not run. I have checked for errors and there were none, and checking the scheduler logs (on both pods) it appears no evaluation of the dag run was done in either scheduler instance. in the cli inspecting the next 60 run execution dates indicates there is no problem with the timetable's interpretation
   
   My suspiscion is that a race condition in the `SELECT ... FOR UPDATE` of the database may have prevented either scheduler executing the DAG. Is this possible? Otherwise how can this have happened? 
   
   ### What you think should happen instead
   
   The dag should have run as it always has previously
   
   ### How to reproduce
   
   I wish I knew! I can't work it out.
   
   ### Operating System
   
   RedHat Enterprise Linux 8
   
   ### Versions of Apache Airflow Providers
   
   2.4.3
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   High availability schedulers (n=2) on OpenShift Kubernetes. Postgres database (version 10.21
   
   ### Anything else
   
   This has happened once before, just under a month ago, on the same DAG.
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #28293: Skipping dag runs on established daily DAG

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #28293: Skipping dag runs on established daily DAG
URL: https://github.com/apache/airflow/issues/28293


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28293: Skipping dag runs on established daily DAG

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #28293:
URL: https://github.com/apache/airflow/issues/28293#issuecomment-1345640631

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #28293: Skipping dag runs on established daily DAG

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #28293:
URL: https://github.com/apache/airflow/issues/28293#issuecomment-1366156538

   Duplicate of https://github.com/apache/airflow/issues/27399 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org