You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "mdnncz (via GitHub)" <gi...@apache.org> on 2026/02/03 12:57:15 UTC
[I] Error when running airflow db clean on dag_version table [airflow]
mdnncz opened a new issue, #61390:
URL: https://github.com/apache/airflow/issues/61390
### Apache Airflow version
3.1.6
### If "Other Airflow 3 version" selected, which one?
3.1.3
### What happened?
when running `airflow db clean --clean-before '2025-10-15' --yes --verbose`, error is reported:
```
Checking table task_instance_history
Found 2 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__task_instance_history__20260203124900
Finished Performing Delete
Checking table celery_taskmeta
Found 13 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__celery_taskmeta__20260203124900
Finished Performing Delete
Checking table callback_request
Found 0 rows meeting deletion criteria.
Checking table session
Found 0 rows meeting deletion criteria.
Checking table xcom
Found 12 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__xcom__20260203124900
Finished Performing Delete
Checking table task_instance
Found 17 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__task_instance__20260203124900
Finished Performing Delete
Checking table deadline
Found 0 rows meeting deletion criteria.
Checking table dag_run
Found 3 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_run__20260203124900
Finished Performing Delete
Checking table dag_version
Found 46 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20260203124900
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(0199ce36-3943-7f32-af3b-3976a2646dc8) is still referenced from table "task_instance".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 7, in <module>
sys.exit(main())
^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line 55, in main
args.func(args)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 114, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/db_command.py", line 296, in cleanup_tables
run_cleanup(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py", line 100, in wrapper
return func(*args, session=session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db_cleanup.py", line 541, in run_cleanup
_cleanup_table(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db_cleanup.py", line 372, in _cleanup_table
_do_delete(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db_cleanup.py", line 262, in _do_delete
raise e
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db_cleanup.py", line 258, in _do_delete
session.execute(delete)
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2351, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2258, in _execute_internal
result = conn.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(0199ce36-3943-7f32-af3b-3976a2646dc8) is still referenced from table "task_instance".
[SQL: DELETE FROM dag_version USING _airflow_deleted__dag_version__20260203124900 WHERE dag_version.id = _airflow_deleted__dag_version__20260203124900.id]
(Background on this error at: https://sqlalche.me/e/20/gkpj)
```
### What you think should happen instead?
no error shall be reported
### How to reproduce
i can provide postgresql db backup as it is development env
### Operating System
windows11/docker desktop/official airflow linux image
### Versions of Apache Airflow Providers
irrelevant
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "mdnncz (via GitHub)" <gi...@apache.org>.
mdnncz commented on issue #61390:
URL: https://github.com/apache/airflow/issues/61390#issuecomment-3841547235
Found the logical issue after using `AIRFLOW__LOGGING__LOGGING_LEVEL=DEBUG` and seing queries issued against the db.
The "old rows query" is invalid for the `dag_version` table.
It has only simple `WHERE base.created_at < :created_at_1` condition. But you can't delete version, which has task_intances (started after the version has been created obviously).
IMHO the query should be:
```
SELECT base.*
FROM dag_version as base
WHERE base.created_at < :created_at_1
AND base.id NOT IN (SELECT DISTINCT dag_version_id FROM task_instance)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "mdnncz (via GitHub)" <gi...@apache.org>.
mdnncz commented on issue #61390:
URL: https://github.com/apache/airflow/issues/61390#issuecomment-3841169121
There is similar issue https://github.com/apache/airflow/issues/52147. But even when running:
```
airflow db clean -t task_instance --clean-before '2025-10-15' --yes
airflow db clean -t dag_version --clean-before '2025-10-15' --yes
```
It does not help and the same error is reported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #61390:
URL: https://github.com/apache/airflow/issues/61390#issuecomment-3842897909
Related https://github.com/apache/airflow/issues/59474
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #61390:
URL: https://github.com/apache/airflow/issues/61390#issuecomment-3841158692
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "mdnncz (via GitHub)" <gi...@apache.org>.
mdnncz commented on issue #61390:
URL: https://github.com/apache/airflow/issues/61390#issuecomment-3846007221
Indeed this is duplicate. Thanks @tirkarthi .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error when running airflow db clean on dag_version table [airflow]
Posted by "mdnncz (via GitHub)" <gi...@apache.org>.
mdnncz closed issue #61390: Error when running airflow db clean on dag_version table
URL: https://github.com/apache/airflow/issues/61390
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org