You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Blizzke (via GitHub)" <gi...@apache.org> on 2023/03/01 08:27:56 UTC
[GitHub] [airflow] Blizzke opened a new issue, #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Blizzke opened a new issue, #29828:
URL: https://github.com/apache/airflow/issues/29828
### Apache Airflow version
2.5.1
### What happened
When making an while declaring a DAG Param, any resulting `jsonschema.exceptions.SchemaErrors` are not shown in the GUI as a DAG import error, nor visible via `airflow dags list-import-errors` because the exception is not caught in the `models/dagbag.py` ` _process_modules`-function (only `ParamValidationError` is)
### What you think should happen instead
The function should not only capture `ParamValidationError`s, but also `jsonschema.exceptions.SchemaError`. and let you know why the DAG is not being imported.
### How to reproduce
Add a `Param` to your dag, but use a wrong configuration. In our case the developer mistakingly used `schema` instead of `type`.
```
"release": Param(
default=None,
description="Competent release to run the update for",
schema="string"
)
```
The `schema` line should've been `type=["string", "null"]`.
### Operating System
Debian
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
Original airflow docker container (2.5.1 - python 3.10)
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Blizzke commented on issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Posted by "Blizzke (via GitHub)" <gi...@apache.org>.
Blizzke commented on issue #29828:
URL: https://github.com/apache/airflow/issues/29828#issuecomment-1449553257
Screenshot of the output during `airflow db upgrade` (only place where it was shown that I could find during debugging this):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #29828:
URL: https://github.com/apache/airflow/issues/29828#issuecomment-1452699881
Fixed in #29056 then. Closing as essentially duplicate of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
URL: https://github.com/apache/airflow/issues/29828
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SamWheating commented on issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Posted by "SamWheating (via GitHub)" <gi...@apache.org>.
SamWheating commented on issue #29828:
URL: https://github.com/apache/airflow/issues/29828#issuecomment-1452638928
This was inadvertently fixed in https://github.com/apache/airflow/pull/29056, where the `Except` in `_process_modules` was modified to catch any exceptions.
Can confirm that these issues are correctly surfaced when running Airflow from `main`:
<img width="1881" alt="image" src="https://user-images.githubusercontent.com/16950874/222573097-6dffd0e7-1acf-4454-95da-0eb305c88abe.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #29828: JSON Schema Validation errors for Params prevent DAG from importing and errors are not shown
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #29828:
URL: https://github.com/apache/airflow/issues/29828#issuecomment-1449549812
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org