You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kaveti (via GitHub)" <gi...@apache.org> on 2024/10/17 13:22:02 UTC

[I] Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks [airflow]

kaveti opened a new issue, #43117:
URL: https://github.com/apache/airflow/issues/43117

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.5.3
   
   ### What happened?
   
   I have dag which dynamically generates close to 800 tasks. when i hit /tries?day=30 and /duration?days=30 webserver API. 
   
   it just simply fails with error. 
   
   - Gunicorn Workers: Increasing the number of workers has not resolved the page loading issues, as the problem isn’t related to incoming request traffic.
   - Timeout Settings: Adjusted both WORKER_TIMEOUT (3000) and master WORKER_TIMEOUT(3000) to higher values, but the page still fails to load.
   - Worker Class: Switching to the gevent worker class did not improve performance, despite its ability to handle parallelism.
   - CPU Limits: Increased the webserver’s CPU limits to 24 cores, yet one of the Gunicorn worker consistently maxes out CPU usage.
   
   
   note: I have checked the webserver_error.log as well. no worker is getting killed abruptly.
   
   
   
   <img width="1328" alt="Screenshot 2024-10-16 at 9 21 40 PM" src="https://github.com/user-attachments/assets/128f627f-0f2a-473b-adda-2716bc5624aa">
   
   
   ### What you think should happen instead?
   
   _No response_
   
   ### How to reproduce
   
   
   - Create a dag which generation dynamic tasks more than 800. and then hit /dag_id/tries?day=30 or /dag_id/duration?day=30 API. 
   
   
   
   
   
   <img width="1707" alt="Screenshot 2024-10-17 at 6 50 31 PM" src="https://github.com/user-attachments/assets/4316359c-bcb9-4227-933a-7bd1902559d9">
   
   
   ### Operating System
   
   Red Hat Enterprise Linux (9.4) - rhel
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #43117: Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks
URL: https://github.com/apache/airflow/issues/43117


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #43117:
URL: https://github.com/apache/airflow/issues/43117#issuecomment-2419536711

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #43117:
URL: https://github.com/apache/airflow/issues/43117#issuecomment-2420687528

   There were lots of similar issues and optimisations implemented since 2.5.3. I recommend you to upgrade it to later version (2.5.3 is released 1.5 year ago and 2.10.2 has proably more than 1000 fixes since).
   
   Please upgrade and try again.
   
   Also if after upgrade you see the same issue - in order to see what happens, some anaysis of what happens in the proces would be useful. Running `pyspy` on the process of gunicorn could bring more issues. Until we get some of those details - converting it into a discussion, maybe someone with similar behaviour could chime in.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org