Re: [PR] Fix scheduler DBDagBag unbounded cache [airflow]

["jason810496 (via GitHub)" <gi...@apache.org>]

jason810496 commented on code in PR #71704:
URL: https://github.com/apache/airflow/pull/71704#discussion_r3796856310


##########
airflow-core/src/airflow/config_templates/config.yml:
##########
@@ -2642,6 +2653,36 @@ scheduler:
       type: integer
       example: ~
       default: "60"
+    dag_cache_size:
+      description: |
+        Max number of deserialized SerializedDAG objects the scheduler keeps in memory, keyed by
+        Dag version ID. Set to 0 for no size limit, leaving eviction to ``dag_cache_ttl``.
+        Set both this and ``dag_cache_ttl`` to 0 to use an unbounded dict with no eviction,
+        matching the behavior before 3.3.2. The cache then grows with the number of Dag versions
+        the scheduler has ever seen, which on a long-running scheduler is unbounded.
+      version_added: 3.3.2

Review Comment:
   I considered this as patch level to escalate the critical scheduler issue quick. Additionally, the version and the behavior are clearly wrote in the docs.
   
   I agreed with what we discussed offline, it still depending on the release manager for whether consider it as patch or minor release. I will keep it as-is for now and wait for further comment, thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org