You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/11 19:00:57 UTC
[GitHub] [airflow] collinmcnulty opened a new issue #17559: Dag Import Error Hidden for existing dags leading to nondescriptive UI errors
collinmcnulty opened a new issue #17559:
URL: https://github.com/apache/airflow/issues/17559
Python version: 3.7.10
**Apache Airflow version**: 2.1.1
**Apache Airflow Provider versions** (please include all providers that are relevant to your bug): None
**Kubernetes version (if you are using kubernetes)**: version.Info{Major:"1", Minor:"18+", GitVersion:"v1.18.19-gke.1701", GitCommit:"d7cecefb99b58e8968f59b59d76448eb1e6ea403", GitTreeState:"clean", BuildDate:"2021-06-23T21:51:59Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}
**Environment**:
- **Cloud provider or hardware configuration**: GCP
- **OS** (e.g. from /etc/os-release): Debian
**What happened**:
We upgraded our airflow from 1.10.14 to 2.1.1 (not sure if the recent upgrade is relevant). Several dags already existed in the metadata db at the time of the upgrade. After the upgrade, some of the dags would give the following error messages if you clicked on Code view or Graph view.
Graph view
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/site-packages/airflow/www/auth.py", line 34, in decorated
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www/decorators.py", line 97, in view_func
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www/decorators.py", line 60, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 2231, in graph
nodes = task_group_to_dict(dag.task_group)
File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 233, in task_group_to_dict
task_group_to_dict(child) for child in sorted(task_group.children.values(), key=lambda t: t.label)
AttributeError: 'NoneType' object has no attribute 'children'
```
Code view
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/site-packages/airflow/www/auth.py", line 34, in decorated
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 905, in code
wrapped=conf.getboolean('webserver', 'default_wrap'),
File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", line 443, in render_template
**kwargs,
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/baseviews.py", line 288, in render_template
template, **dict(list(kwargs.items()) + list(self.extra_args.items()))
File "/usr/local/lib/python3.7/site-packages/flask/templating.py", line 140, in render_template
ctx.app,
File "/usr/local/lib/python3.7/site-packages/flask/templating.py", line 120, in _render
rv = template.render(context)
File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/local/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.7/site-packages/airflow/www/templates/airflow/dag_code.html", line 20, in top-level template code
{% extends "airflow/dag.html" %}
File "/usr/local/lib/python3.7/site-packages/airflow/www/templates/airflow/dag.html", line 21, in top-level template code
{% from 'appbuilder/dag_docs.html' import dag_docs %}
File "/usr/local/lib/python3.7/site-packages/astronomer/airflow/version_check/templates/astro-baselayout.html", line 1, in top-level template code
{% extends airflow_base_template %}
File "/usr/local/lib/python3.7/site-packages/airflow/www/templates/airflow/main.html", line 20, in top-level template code
{% extends 'appbuilder/baselayout.html' %}
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 2, in top-level template code
{% import 'appbuilder/baselib.html' as baselib %}
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 37, in top-level template code
{% block body %}
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 19, in block "body"
{% block content %}
File "/usr/local/lib/python3.7/site-packages/airflow/www/templates/airflow/dag_code.html", line 30, in block "content"
{{ super() }}
File "/usr/local/lib/python3.7/site-packages/airflow/www/templates/airflow/dag.html", line 139, in block "content"
<input type="hidden" name="origin" value="{{ url_for('Airflow.' + dag.default_view, dag_id=dag.dag_id) }}">
TypeError: can only concatenate str (not "NoneType") to str
```
We tried deleting one of the affected dags with the "Delete Dag" button and then when the dag file was re-parsed, it through a dag import error, which was helpful.
**What you expected to happen**:
Get the dag import error immediately, instead of allowing me to click on the dag and get nondescriptive error message.
**How to reproduce it**:
1. Write a valid dag
2. Have it parsed by an Airflow instance (might need to be pre-2.0)
3. Upload a new, broken version of the dag file.
4. (Unclear if necessary) Upgrade to airflow 2.1.1
5. In the UI, go to the dag and click on Code or Graph view.
**Anything else we need to know**:
It's possible that this problem was affecting our 1.10.14 instance as well. These dags were only checked as part of verifying the upgrade and had not run or been looked at since the change that caused the import error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] michaelswierszcz commented on issue #17559: Dag Import Error Hidden for existing dags leading to nondescriptive UI errors
Posted by GitBox <gi...@apache.org>.
michaelswierszcz commented on issue #17559:
URL: https://github.com/apache/airflow/issues/17559#issuecomment-933955842
Same error, deleted dag and reprocessed to resolve
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] collinmcnulty commented on issue #17559: Dag Import Error Hidden for existing dags leading to nondescriptive UI errors
Posted by GitBox <gi...@apache.org>.
collinmcnulty commented on issue #17559:
URL: https://github.com/apache/airflow/issues/17559#issuecomment-924259267
@ephraimbuddy Confirmed. Tested this on main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy closed issue #17559: Dag Import Error Hidden for existing dags leading to nondescriptive UI errors
Posted by GitBox <gi...@apache.org>.
ephraimbuddy closed issue #17559:
URL: https://github.com/apache/airflow/issues/17559
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on issue #17559: Dag Import Error Hidden for existing dags leading to nondescriptive UI errors
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #17559:
URL: https://github.com/apache/airflow/issues/17559#issuecomment-924161017
Fixed in https://github.com/apache/airflow/pull/17818. @collinmcnulty we now show import errors on dag views, please verify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org