You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/06 21:57:23 UTC
[GitHub] [airflow] ashb opened a new pull request #12866: Show DAG serialization errors in the UI.
ashb opened a new pull request #12866:
URL: https://github.com/apache/airflow/pull/12866
The previous behaviour led to "bad" data being written in the DB -- for
example:
```json
"dag": {
"tasks": [
"serialization_failed"
],
```
(`tasks` should be a list of dictionaries. It clearly isn't.)
Instead of doing this we throw an error, that is captured and showing
using the existing import_error mechanism for DAGs. This almost
certainly happens because a user has done "something interesting".
Example of error
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #12866: Show DAG serialization errors in the UI.
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #12866:
URL: https://github.com/apache/airflow/pull/12866
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12866: Show DAG serialization errors in the UI.
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12866:
URL: https://github.com/apache/airflow/pull/12866#discussion_r537134087
##########
File path: airflow/exceptions.py
##########
@@ -130,6 +130,10 @@ class SerializedDagNotFound(DagNotFound):
"""Raise when DAG is not found in the serialized_dags table in DB"""
+class SerializationError(AirflowException):
+ """A problem occurred when trying to serialized a DAG"""
Review comment:
```suggestion
"""A problem occurred when trying to serialize a DAG"""
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12866: Show DAG serialization errors in the UI.
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12866:
URL: https://github.com/apache/airflow/pull/12866#discussion_r537134102
##########
File path: airflow/models/dagbag.py
##########
@@ -517,6 +517,27 @@ def sync_to_db(self, session: Optional[Session] = None):
from airflow.models.dag import DAG
from airflow.models.serialized_dag import SerializedDagModel
+ def _serialze_dag_capturing_errors(dag, session):
+ """
+ Try to serialized the dag to the DB, but make a note of any errors.
Review comment:
```suggestion
Try to serialize the dag to the DB, but make a note of any errors.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org