You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amit-mittal (via GitHub)" <gi...@apache.org> on 2025/07/10 20:08:41 UTC

[I] Disabling auto-refresh from UI has gone missing [airflow]

amit-mittal opened a new issue, #53157:
URL: https://github.com/apache/airflow/issues/53157

   ### Apache Airflow version
   
   3.0.2
   
   ### If "Other Airflow 2 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
   After upgrading from Airflow v2 to v3, I am not able to see (👀) a button to disable "auto-refresh" of UI. I looked multiple times making sure I am not missing it, but I am for sure not able to see it.
   
   ### What you think should happen instead?
   
   - Disable auto-refresh from UI
   - As an admin, ideally also disable auto-refresh for everyone.
   
   ### How to reproduce
   
   Upgrade to Airflow v3 and I don't see a toggle like in the image anywhere
   
   <img width="254" height="86" alt="Image" src="https://github.com/user-attachments/assets/dc372fc7-3ad5-4362-bdca-823dbc5851d7" />
   
   ### Operating System
   
   Official Airflow docker image
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "amit-mittal (via GitHub)" <gi...@apache.org>.
amit-mittal commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3059133766

   I think since upgrading to Airflow v3, we are seeing those continuous polls have made UI less responsive and putting more load on the [backend DB](https://github.com/apache/airflow/issues/53154). From UX perspective, I would love to keep auto-refresh enabled for our users though.
   
   We are seeing `Task Instance` page taking > 3 minutes to load. We cleaned up the DB history to keep only 2 years worth of data, but looks like we have to do even more aggressively to make things better. We can't go less than a year because of compliance needs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3058967497

   I think - with reactive approach, it's been a deliberate decision to get rid of the old "no-refresh" behaviour. Is there any particular reason why you want to bring it back? What's the reasoning and need ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3059278919

   I think if you are seeing such long loads, there is something seriously wrong  with your setup - for example you gave too little resources to your db or api server. I suggest you look at what is taking so much time - all queries should be ussing indexes and pagination to make things faster. I f you can track it down _ by i section g what is going wrong - which api calls are taking long and looking at corresponding queries in the dn and look at your resource usage you might find the culprit. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #53157: Disabling auto-refresh from UI has gone missing
URL: https://github.com/apache/airflow/issues/53157


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "amit-mittal (via GitHub)" <gi...@apache.org>.
amit-mittal commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3059552943

   I do agree with you that it shouldn't take that long and there is definitely something wrong with our setup. In pre-prod environments which we have currently upgrade to v3, we allocate less resources, but still it shouldn't have been this slow. Regarding the index, I linked you to the GitHub issue in my previous message where we clearly are not using the index as part of scheduler loop.
   
   I enabled `otel` tracing, but I couldn't find any traces for the critical request from WebServer (I do see traces for Scheduler). Is there a way to get more details about this? Sorry, I couldn't understand what you meant by "i section g what is going wrong - which api calls are taking long and looking at corresponding queries in the dn".
   
   Some details about our setup:
   - We are running MySQL 8.x with 16 GB memory and 10 GB InnoDB buffer pool size. Our CPU usage used to be at 10%, but now it hovers at 50%.
   - The instance handles around 200 DAGs, not too complicated. In database, we have last 2 years history.
   - We have set page limit in Airflow as 100 
   - AirflowWebServer has 3 GB memory and can go max up to 8 GB
   - AirflowScheduler has 4 GB memory and can go max up to 10 GB. We use `LocalExecutor`.
   - AirflowDAGProcessor has 2 GB memory and can go max up to 5 GB.
   
   Production instance has more resources, but currently the instance running on Airflow v3.0.2 has the above resources. If you have a recommendation (or need more details), we can increase the allocated resources and tweak the Airflow configurations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3058962675

   Why?  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Disabling auto-refresh from UI has gone missing [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #53157:
URL: https://github.com/apache/airflow/issues/53157#issuecomment-3061473850

   > Production instance and DBs running Airflow v2.10.x have more resources, but currently the instance running on Airflow v3.0.2 has the above resources. If you have a recommendation (or need more details), that will be of huge help. We can increase the allocated resources and tweak the Airflow configurations.
   
   Unfortunately - I have no time to debug your setup and configuration - you have to remember that here we help people in our free time.
   
   Not sure which Github issue you are talking about - I look at about few hundred issues a day - and it's not very likely I will be helping with that issue. But if refreshing a page takes 3 minutes then you should definitely explain that in the issue including all sql queries and index information you found and explain that this is what happens, because that obviously means that **something** is wrong. But definitely "disabling auto refresh" is not a solution to the problem. 
   
   I convert it to a discussion now and if you have the other issue opened with all the details, then I think you should continue conversation there. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org