You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kalluripradeep (via GitHub)" <gi...@apache.org> on 2026/06/21 09:38:52 UTC
[I] MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted [airflow]
kalluripradeep opened a new issue, #68794:
URL: https://github.com/apache/airflow/issues/68794
### Apache Airflow version
main
### What happened
`[state_store] default_retention_days` defaults to 30 and is documented as:
"Number of days to retain task state after their last update. Rows older than
this are removed when cleanup is triggered."
However, `MetastoreBackend.cleanup()` only deletes rows where `expires_at < now()`.
Task workers always write with `expires_at=None`, so those rows are never cleaned up
regardless of the `default_retention_days` setting. The `task_state_store` table
grows unbounded.
The model comment, CLI description, and config docs all promise a
"global updated_at + default_retention_days check" that does not exist in the code.
### What you expected to happen
Rows with `expires_at=NULL` older than `default_retention_days` days should be
deleted when `airflow state-store cleanup` is run.
### How to reproduce
1. Set `[state_store] default_retention_days = 30`
2. Run tasks that write to task state store (expires_at=None by default)
3. Wait 30+ days (or manually backdate `updated_at`)
4. Run `airflow state-store cleanup`
5. Rows are NOT deleted despite exceeding the retention window
### Additional context
Relevant code: `airflow-core/src/airflow/state/metastore.py`, `cleanup()` method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted [airflow]
Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 commented on issue #68794:
URL: https://github.com/apache/airflow/issues/68794#issuecomment-4765039666
cc: @amoghrajesh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted [airflow]
Posted by "kalluripradeep (via GitHub)" <gi...@apache.org>.
kalluripradeep commented on issue #68794:
URL: https://github.com/apache/airflow/issues/68794#issuecomment-4766078468
Closing as **not a bug**.
The initial report was based on a wrong assumption: that task workers write `expires_at=None` to the DB. They don't.
`TaskStateStoreAccessor.set()` in the task SDK resolves `expires_at` **client-side on the worker** before sending to the server. With the default `retention=None` and `default_retention_days=30`, workers write `expires_at = now + 30 days`. The existing `WHERE expires_at < now()` cleanup in `MetastoreBackend.cleanup()` handles those rows correctly after 30 days.
`expires_at=NULL` in the DB is only written when `retention=NEVER_EXPIRE` is explicitly passed, or when `default_retention_days=0` (retention disabled). Both cases are intentional and must not be deleted by the cleanup pass.
Thanks to @amoghrajesh for the correction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh closed issue #68794: MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted
URL: https://github.com/apache/airflow/issues/68794
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MetastoreBackend.cleanup() ignores default_retention_days — task state rows with null expires_at are never deleted [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #68794:
URL: https://github.com/apache/airflow/issues/68794#issuecomment-4765209163
**The premise in step 2 is incorrect**,`task_state_store.set(key, value)` with no explicit `retention` does not write `expires_at=NULL`. The task sdk resolves `expires_at` at write time: https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/execution_time/context.py#L587-L593
With the default `default_retention_days = 30`, workers write `expires_at = now + 30 days`. cleanup() correctly checks for `WHERE expires_at < now` and correctly removes those rows after the window elapses.
For future reports: please validate reproduction steps end to end before filing an issue. Inspecting the actual `expires_at` column values in `task_state_store` after running tasks would have shown the field is already populated correctly.
Closing as invalid.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org