Re: [I] Airflow DAG version keeps increasing [airflow]

["sbrinkhorst (via GitHub)" <gi...@apache.org>]

sbrinkhorst commented on issue #58310:
URL: https://github.com/apache/airflow/issues/58310#issuecomment-5193506705

   We recently migrated to Airflow 3.2 (from an Airflow 2 version) and I was surprised by the high version number of some DAGs but didn't pay much attention to it. After migrating to 3.3 we found out that it is actually a problem: if you go back to older DagRuns it crashes our API pods because it goes out of memory. We think this because Airflow 3.3 fixes the issue of not correctly seeing the run in the grid view for older DAGs. We parameterize our DAGs heavily and run a lot, so going back to an older run of a DAG that has 74k runs and 54k versions killed the pods. And lo and behold, it is indeed only for DAGs that have some random element in it.
   
   Did you do something to clean up the table, or should it be enough that we have database cleanup configured? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org