You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "HsiuChuanHsu (via GitHub)" <gi...@apache.org> on 2025/10/11 08:38:50 UTC
Re: [I] ForeignKeyViolation error when doing `airflow db clean` [airflow]
HsiuChuanHsu commented on issue #56192:
URL: https://github.com/apache/airflow/issues/56192#issuecomment-3393068953
Hi @hket-ianchu,
I was able to reproduce the issue you provided. Below are the reproduction results:
**Create testing data in `dag_version` & `task_instance` table.**
- `dag_version`: created_at | 2025-09-01
<img width="477" height="158" alt="Image" src="https://github.com/user-attachments/assets/f0395a14-58f1-4e08-942d-478a8e58f8de" />
- `task_instance-1`: start_date | 2025-09-06
<img width="434" height="666" alt="Image" src="https://github.com/user-attachments/assets/be676f8e-35d7-43ea-a6c4-84725c88903d" />
- `task_instance-2`: start_date | 2025-10-01
<img width="435" height="652" alt="Image" src="https://github.com/user-attachments/assets/934d4091-2c35-4549-b37f-0a1c3a11162e" />
---
**Use `airflow db clean` to clean up the records**
```bash
airflow db clean --clean-before-timestamp "2025-09-21T10:30" --skip-archive --yes
```
**Result**
```bash
airflow db clean --clean-before-timestamp "2025-09-21T10:30" --skip-archive --yes
/opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py:40 DeprecationWarning: airflow.exceptions.AirflowDagCycleException is deprecated. Use airflow.sdk.exceptions.AirflowDagCycleException instead.
2025-10-11T08:33:45.238607Z [warning ] Table celery_tasksetmeta not found. Skipping. [airflow.utils.db_cleanup] loc=db_cleanup.py:555
2025-10-11T08:33:45.238729Z [warning ] Table celery_taskmeta not found. Skipping. [airflow.utils.db_cleanup] loc=db_cleanup.py:555
Checking table task_instance_history
Found 0 rows meeting deletion criteria.
Checking table xcom
Found 0 rows meeting deletion criteria.
Checking table task_instance
Found 1 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__task_instance__20251011083345
Finished Performing Delete
Checking table deadline
Found 0 rows meeting deletion criteria.
Checking table dag_run
Found 1 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_run__20251011083345
Finished Performing Delete
Checking table dag_version
Found 1 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20251011083345
Traceback (most recent call last):
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(018c5e5e-7a8b-7000-8000-000000000001) is still referenced from table "task_instance".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/python/bin/airflow", line 10, in <module>
sys.exit(main())
^^^^^^
File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 55, in main
args.func(args)
File "/opt/airflow/airflow-core/src/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/airflow-core/src/airflow/utils/cli.py", line 115, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/opt/airflow/airflow-core/src/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/airflow-core/src/airflow/cli/commands/db_command.py", line 297, in cleanup_tables
run_cleanup(
File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 100, in wrapper
return func(*args, session=session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 544, in run_cleanup
_cleanup_table(
File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 371, in _cleanup_table
_do_delete(
File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 261, in _do_delete
raise e
File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 257, in _do_delete
session.execute(delete)
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1717, in execute
result = conn._execute_20(statement, params or {}, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self._handle_dbapi_exception(
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
util.raise_(
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(018c5e5e-7a8b-7000-8000-000000000001) is still referenced from table "task_instance".
[SQL: DELETE FROM dag_version USING _airflow_deleted__dag_version__20251011083345 WHERE dag_version.id = _airflow_deleted__dag_version__20251011083345.id]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org