You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/24 15:03:12 UTC
[GitHub] [airflow] zealpatel1990 opened a new issue #17814: Last Dagrun is not working.
zealpatel1990 opened a new issue #17814:
URL: https://github.com/apache/airflow/issues/17814
<!--
Welcome to Apache Airflow!
Please complete the next sections or the issue will be closed.
-->
**Apache Airflow version**:2.1.3
<!-- AIRFLOW VERSION IS MANDATORY -->
**OS**: Running Airflow in docker container.
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
<!-- MANDATORY! You can get it via `cat /etc/oss-release` for example -->
**Apache Airflow Provider versions**:
apache-airflow-providers-amazon==2.1.0
apache-airflow-providers-celery==2.0.0
apache-airflow-providers-cncf-kubernetes==2.0.1
apache-airflow-providers-docker==2.1.0
apache-airflow-providers-elasticsearch==2.0.2
apache-airflow-providers-ftp==2.0.0
apache-airflow-providers-google==5.0.0
apache-airflow-providers-grpc==2.0.0
apache-airflow-providers-hashicorp==2.0.0
apache-airflow-providers-http==2.0.0
apache-airflow-providers-imap==2.0.0
apache-airflow-providers-microsoft-azure==3.1.0
apache-airflow-providers-mysql==2.1.0
apache-airflow-providers-postgres==2.0.0
apache-airflow-providers-redis==2.0.0
apache-airflow-providers-sendgrid==2.0.0
apache-airflow-providers-sftp==2.1.0
apache-airflow-providers-slack==4.0.0
apache-airflow-providers-sqlite==2.0.0
apache-airflow-providers-ssh==2.1.0
<!-- You can use `pip freeze | grep apache-airflow-providers` (you can leave only relevant ones)-->
**Deployment**: docker-compose
<!-- e.g. Virtualenv / VM / Docker-compose / K8S / Helm Chart / Managed Airflow Service -->
<!-- Please include your deployment tools and versions: docker-compose, k8s, helm, etc -->
**What happened**: last_dagtuns are not showing up. Here is the error:
```
Python version: 3.6.14
Airflow version: 2.1.3
Node: 078f1a56398c
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/airflow/.local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/auth.py", line 34, in decorated
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/views.py", line 858, in last_dagruns
for r in query
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/views.py", line 858, in <dictcomp>
for r in query
AttributeError: 'NoneType' object has no attribute 'isoformat'
```
<!-- Please include exact error messages if you can -->
**What you expected to happen**: It should show me last Dagrun date time
<!-- What do you think went wrong? -->
**How to reproduce it**: Run latest 2.1.3 airflow on Ubuntu in docker. Turn off examples and create your own DAG.
<!--
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
You can include images/screen-casts etc. by drag-dropping the image here.
-->
**Anything else we need to know**: I haven't seen this last run working ever since I have started using Aiflow
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here inside fenced
``` ``` blocks or inside a foldable details tag if it's long:
<details><summary>x.log</summary> lots of stuff </details>
-->
**Are you willing to submit a PR?**: Sure! but pretty new to Airflow. I believe there should be condition to check for NoneType before accessing its attributes.
<!---
This is absolutely not required, but we are happy to guide you in contribution process
especially if you already have a good understanding of how to implement the fix.
Airflow is a community-managed project and we love to bring new contributors in.
Find us in #airflow-how-to-pr on Slack!
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] david30907d commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
david30907d commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-910038009
bumped into the same error, thanks @uranusjr for the quick fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-910091376
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bhavaniravi commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
bhavaniravi commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-929074990
I could recreate this issue locally and found a few more insights
Ran airflow with 5 DAGs(non-example)
1. when no dags are active, the `last_dagrun` API returns an empty dict. The loading issue is only on the UI
2. when the 1st DAG is activated, the `last_dagrun` sends an error until the task is completed
3. 1st DAG ran successfully, and the last run was populated. UI is no longer loading
4. There was a case where subdag just hung for a long time after a dagrun was created, the API returned 500 error then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] david30907d commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
david30907d commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-910038009
bumped into the same error, thanks @uranusjr for the quick fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
uranusjr closed issue #17814:
URL: https://github.com/apache/airflow/issues/17814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] argzeal removed a comment on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
argzeal removed a comment on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-904917308
Thanks @uranusjr for quick fix. Now I understand that I have DAG which schedule is set to `None`, that also breaks last run of other dags as well. As soon as DAG is manually triggered at least once it works. I will create PR as soon as I can to accommodate the suggested change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] argzeal commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
argzeal commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-904917308
Thanks @uranusjr for quick fix. Now I understand that I have DAG which schedule is set to `None`, that also breaks last run of other dags as well. As soon as DAG is manually triggered at least once it works. I will create PR as soon as I can to accommodate the suggested change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zealpatel1990 commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
zealpatel1990 commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-904919140
Thanks @uranusjr for quick fix. Now I understand that I have DAG which schedule is set to None, that also breaks last run of other dags as well. As soon as DAG is manually triggered at least once it works. I will create PR as soon as I can to accommodate the suggested change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bhavaniravi commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
bhavaniravi commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-929074990
I could recreate this issue locally and found a few more insights
Ran airflow with 5 DAGs(non-example)
1. when no dags are active, the `last_dagrun` API returns an empty dict. The loading issue is only on the UI
2. when the 1st DAG is activated, the `last_dagrun` sends an error until the task is completed
3. 1st DAG ran successfully, and the last run was populated. UI is no longer loading
4. There was a case where subdag just hung for a long time after a dagrun was created, the API returned 500 error then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-904722248
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #17814: Last Dagrun is not working.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #17814:
URL: https://github.com/apache/airflow/issues/17814#issuecomment-904787987
The code is in `airflow/www/views.py`. Search for `def last_dagruns` and the error is toward the bottom of the function.
Change this
```python
resp = {
r.dag_id.replace('.', '__dot__'): {
"dag_id": r.dag_id,
"execution_date": r.execution_date.isoformat(),
"start_date": r.start_date.isoformat(),
}
for r in query
}
```
to something like this:
```python
def _datetime_to_string(value: Optional[DateTime]) -> Optional[str]:
if value is None:
return None
return value.isoformat()
resp = {
r.dag_id.replace('.', '__dot__'): {
"dag_id": r.dag_id,
"execution_date": _datetime_to_string(r.execution_date),
"start_date": _datetime_to_string(r.start_date),
}
for r in query
}
```
I had this fix in another PR (not merged to main yet), but if you could submit a PR just for this change, it can likely be merged pretty quickly and included in 2.1.4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org