You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "sicarul (via GitHub)" <gi...@apache.org> on 2024/04/15 17:26:24 UTC
[I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
sicarul opened a new issue, #39038:
URL: https://github.com/apache/airflow/issues/39038
### Apache Airflow version
2.9.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Since upgrading from 2.8.3 to 2.9.0, some DAGs are now empty
### What you think should happen instead?
Graphs should show in the UI
### How to reproduce
Some times when i open the graph it shows for a fraction of a second and then disappears, and other times i see it completely empty; I'm not sure how to reproduce it, but when i open the dev console i see this, which mentions a shape and then names a model *that is not part of this dag*, which seems even weirder. The dag has a task group if that's relevant:
```
Error: org.eclipse.elk.graph.json.JsonImportException: Referenced shape does not exist: dbt_models.stg_mysql_single_user_organization
```
<img width="1913" alt="image" src="https://github.com/apache/airflow/assets/4952116/c1a7edc1-1735-4f35-9db4-c3208deeab3d">
In case it's relevant, this dag is generated from a single python file that generates dozens of dags from a yaml, not sure if they can somehow be
### Operating System
Debian GNU/Linux 12 (bookworm)
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon[aiobotocore]==8.20.0
apache-airflow-providers-common-sql==1.12.0
apache-airflow-providers-http==4.10.1
apache-airflow-providers-snowflake==5.4.0
apache-airflow-providers-sqlite==3.7.1
apache-airflow-providers-redis==3.6.1
apache-airflow-providers-celery==3.6.2
### Deployment
Other Docker-based deployment
### Deployment details
Running on Docker images on AWS ECS
### Anything else?
I am willing to submit a PR but i have no idea where/why this happens, and zero knowledge of Airflow's codebase, if someone can at least guide me as to where could i fix this issue
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "sicarul (via GitHub)" <gi...@apache.org>.
sicarul commented on issue #39038:
URL: https://github.com/apache/airflow/issues/39038#issuecomment-2059539545
@bbovenzi Yes, that seems to be the case. My guess is this didn't happen before because Airflow was not drawing datasets in the graph
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "sicarul (via GitHub)" <gi...@apache.org>.
sicarul commented on issue #39038:
URL: https://github.com/apache/airflow/issues/39038#issuecomment-2057479094
If it helps to decipher this problem, i found another case:
<img width="1902" alt="image" src="https://github.com/apache/airflow/assets/4952116/895c1425-5b66-4ecc-9c08-bd7ab563840e">
And in this one, i was able to find old tasks with that name (That task does not exist any more) - However even after deleting the deprecated task instances using the UI, the same error persists in the console, my impression is it seems some "phantom records" are being picked up by the graph library?.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #39038:
URL: https://github.com/apache/airflow/issues/39038#issuecomment-2057449333
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on issue #39038:
URL: https://github.com/apache/airflow/issues/39038#issuecomment-2059323200
So this only happens to DAGs with Datasets? And it sounds like it happens when we retrieve datasets but either the dataset or task no longer exists so we're pointing to a nonexistent node. Sounds straightforward enough to add a check for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #39038: Graph is empty after update to 2.9.0 on some DAGs
URL: https://github.com/apache/airflow/issues/39038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Graph is empty after update to 2.9.0 on some DAGs [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #39038:
URL: https://github.com/apache/airflow/issues/39038#issuecomment-2057736524
cc @bbovenzi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org