jscheffl opened a new issue, #49487:
URL: https://github.com/apache/airflow/issues/49487
### Apache Airflow version
3.0.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
I am attempting to upgrade our 2.10.5 to 3.0.0rc4 and migration fails with:
```
[2025-04-20T21:55:25.758+0000] {migration.py:618} INFO - Running upgrade d03e4a635aa3 -> 2b47dc6bc8df, add dag versioning.
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.NotNullViolation: column "dag_id" of relation "dag_code" contains null values
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line 55, in main
args.func(args)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 48, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 111, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/db_command.py", line 197, in migratedb
run_db_migrate_command(args, db.upgradedb, _REVISION_HEADS_MAP)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/db_command.py", line 125, in run_db_migrate_command
command(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py", line 101, in wrapper
return func(*args, session=session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db.py", line 1142, in upgradedb
command.upgrade(config, revision=to_revision or "heads")
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/command.py", line 408, in upgrade
script.run_env()
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/script/base.py", line 586, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 95, in load_python_file
module = load_module_py(module_id, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 113, in load_module_py
spec.loader.exec_module(module) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/env.py", line 138, in <module>
run_migrations_online()
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/env.py", line 132, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/runtime/environment.py", line 946, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
step.migration_fn(**kw)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py", line 238, in upgrade
with op.batch_alter_table("dag_code") as batch_op:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 144, in __exit__
next(self.gen)
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/operations/base.py", line 397, in batch_alter_table
impl.flush()
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/operations/batch.py", line 115, in flush
fn(*arg, **kw)
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/postgresql.py", line 188, in alter_column
super().alter_column(
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 280, in alter_column
self._exec(
File "/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 246, in _exec
return conn.execute(construct, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/future/engine.py", line 286, in execute
return self._execute_20(
^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1477, in _execute_ddl
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
util.raise_(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) column "dag_id" of relation "dag_code" contains null values
[SQL: ALTER TABLE dag_code ALTER COLUMN dag_id SET NOT NULL]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
```
### What you think should happen instead?
Migration should not fail or report an error what the problem is.
### How to reproduce
Happens in migration env... unfortunately no diagnostic information printed to understand where it fails.
### Operating System
Kubernetes
### Versions of Apache Airflow Providers
not relevant
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Helm chart 1.16.0 on K8s
Deployment based on extended Docker image from 3.0.0rc4, Python 3.12
### Anything else?
Deployment was on Airflow 2.10.5 before.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
jscheffl commented on issue #49487: URL: https://github.com/apache/airflow/issues/49487#issuecomment-2817378261 Note: I was able to "tweak" and bypass the problem via `DELETE FROM dag_code WHERE fileloc_hash NOT IN (SELECT fileloc_hash FROM serialized_dag);` but am not sure what bad side effect this generates. Note that I did this in my dev env only but am hesitant to plan the same for my PROD env. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
jscheffl commented on issue #49487: URL: https://github.com/apache/airflow/issues/49487#issuecomment-2817362898 Note: Looks similar to #49478 but the generated error is different. Therefore opened a separate bug to triage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
jscheffl closed issue #49487: Migration fails in discovering dag_id for adding DAG versioning URL: https://github.com/apache/airflow/issues/49487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
zachliu commented on issue #49487: URL: https://github.com/apache/airflow/issues/49487#issuecomment-2825506675 from 2.10.5 to 3.0.0 ```sql DELETE FROM dag_code WHERE fileloc_hash NOT IN (SELECT fileloc_hash FROM serialized_dag); ``` deleted ~20% of the entries in `dag_code`. they must be relics from the distant past -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
CincyBC commented on issue #49487: URL: https://github.com/apache/airflow/issues/49487#issuecomment-2822279872 I ran into this issue and manually running the workaround resolved it for me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
jscheffl commented on issue #49487: URL: https://github.com/apache/airflow/issues/49487#issuecomment-2817732453 Closing as the workaround is the same resolution like in #49478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org