You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dstandish (via GitHub)" <gi...@apache.org> on 2025/04/20 14:25:51 UTC
[I] Migration fails when same dag id appears in two files [airflow]
dstandish opened a new issue, #49476:
URL: https://github.com/apache/airflow/issues/49476
### Body
From @vatsrahul1001
I debugged this and found the root cause.
The issue occurs when the same DAG exists in two different files. In such cases, the [condition](https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py#L173) `dag_code.fileloc_hash = serialized_dag.fileloc_hash` fails. As a result, the `dag_id` column in the `dag_code` table remains null for that specific DAG, leading to the following error:
```
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) column "dag_id" of relation "dag_code" contains null values.
```
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Migration fails when same dag id appears in two files [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on issue #49476:
URL: https://github.com/apache/airflow/issues/49476#issuecomment-2817224777
The issue is that we still create a dag code record even when there's no serdag from it
dagcode
<img width="762" alt="Image" src="https://github.com/user-attachments/assets/c2f77107-7400-42e1-a8cf-567438725e2b" />
serdag
<img width="857" alt="Image" src="https://github.com/user-attachments/assets/045c3d36-5396-4e87-8b35-c9f343c9c998" />
dag
<img width="698" alt="Image" src="https://github.com/user-attachments/assets/242d0fa6-34a5-4b3a-874d-19acb457f358" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Migration fails when same dag id appears in two files [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish closed issue #49476: Migration fails when same dag id appears in two files
URL: https://github.com/apache/airflow/issues/49476
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Migration fails when same dag id appears in two files [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on issue #49476:
URL: https://github.com/apache/airflow/issues/49476#issuecomment-2817210682
side problem:
if migration fails, and you try to do it again, you'll get this error
```sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table _alembic_tmp_dag_code already exists
[SQL:
CREATE TABLE _alembic_tmp_dag_code (
fileloc_hash BIGINT NOT NULL,
fileloc VARCHAR(2000) NOT NULL,
last_updated TIMESTAMP NOT NULL,
source_code TEXT NOT NULL,
id CHAR(32),
dag_version_id CHAR(32),
source_code_hash VARCHAR(32),
dag_id VARCHAR(250),
created_at TIMESTAMP
)
]
(Background on this error at: https://sqlalche.me/e/14/e3q8)```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org