You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "proofrock (via GitHub)" <gi...@apache.org> on 2023/07/03 15:08:07 UTC
[GitHub] [airflow] proofrock opened a new issue, #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
proofrock opened a new issue, #32332:
URL: https://github.com/apache/airflow/issues/32332
### Apache Airflow version
2.6.2
### What happened
I install Airflow with:
```
pip install apache-airflow['postgresql']
pip install psycopg2-binary
# airflow.cfg: executor = LocalExecutor
# airflow.cfg: sql_alchemy_conn = postgresql+psycopg2://airflow:airflow@localhost/airflow
airflow db init
```
Last week everything was fine. Today, the last command fails with:
```
DB: postgresql+psycopg2://airflow:***@localhost/airflow
/home/adminuser/pyairflow/lib/python3.9/site-packages/pydantic/_internal/_config.py:257 UserWarning: Valid config keys have changed in V2:
* 'orm_mode' has been renamed to 'from_attributes'
Traceback (most recent call last):
File "/home/adminuser/pyairflow/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/__main__.py", line 48, in main
args.func(args)
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/cli/cli_config.py", line 52, in command
return func(*args, **kwargs)
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/cli/commands/db_command.py", line 37, in initdb
db.initdb()
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/utils/session.py", line 76, in wrapper
return func(*args, session=session, **kwargs)
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/utils/db.py", line 711, in initdb
import_all_models()
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/models/__init__.py", line 60, in import_all_models
__getattr__(name)
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/models/__init__.py", line 78, in __getattr__
val = import_string(f"{path}.{name}")
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/utils/module_loading.py", line 36, in import_string
module = import_module(module_path)
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/models/skipmixin.py", line 25, in <module>
from airflow.serialization.pydantic.dag_run import DagRunPydantic
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/serialization/pydantic/dag_run.py", line 24, in <module>
from airflow.serialization.pydantic.dataset import DatasetEventPydantic
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/airflow/serialization/pydantic/dataset.py", line 40, in <module>
class TaskOutletDatasetReferencePydantic(BaseModelPydantic):
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 95, in __new__
private_attributes = inspect_namespace(
File "/home/adminuser/pyairflow/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py", line 328, in inspect_namespace
raise PydanticUserError(
pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = <class 'str'>`. All model fields require a type annotation; if `dag_id` is not meant to be a field, you may be able to resolve this error by annotating it as a `ClassVar` or updating `model_config['ignored_types']`.
For further information visit https://errors.pydantic.dev/2.0/u/model-field-missing-annotation
```
### What you think should happen instead
The db init used to complete without any issue, on a clean Debian VM just like the one I'm testing on.
### How to reproduce
In a fresh Azure VM with Debian 11 I use this script:
```
sudo -u postgres psql -c "CREATE USER airflow PASSWORD 'airflow';"
sudo -u postgres psql -c "CREATE DATABASE airflow;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO airflow;"
python3 -m venv ~/pyairflow
echo 'alias afpip="~/pyairflow/bin/pip3"' >> ~/.bash_profile
echo 'alias afpython="~/pyairflow/bin/python3"' >> ~/.bash_profile
source ~/.bash_profile
afpip install apache-airflow['postgresql']
afpip install psycopg2-binary
airflow > /dev/null 2>&1
mv ~/airflow/airflow.cfg ~/airflow/airflow.cfg.orig
cat ~/airflow/airflow.cfg.orig | sed -e 's/^executor =.*$/executor = LocalExecutor/g' | sed -e 's/^sql_alchemy_conn =.*$/sql_alchemy_conn = postgresql+psycopg2:\/\/airflow:airflow@localhost\/airflow/g' > ~/airflow/airflow.cfg
airflow db init
```
### Operating System
Linux Debian 11, an Azure VM
### Versions of Apache Airflow Providers
apache-airflow-providers-common-sql==1.5.2
apache-airflow-providers-ftp==3.4.2
apache-airflow-providers-http==4.4.2
apache-airflow-providers-imap==3.2.2
apache-airflow-providers-sqlite==3.4.2
### Deployment
Virtualenv installation
### Deployment details
As above
### Anything else
Same problem should be reported [here](https://www.reddit.com/r/apache_airflow/comments/14oji77/airflow_262_and_pydantic_warningserrors/).
I am not an expert in python :-(
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1619732778
Surely. If you have proposal where you'd look for it and make a PR for that, that would be great.
We have a nice system where you can just click 'Suggest a change in this page' button in our docs, and you will get a PR opened and you can add a doc change without leaving the GitHub UI. And you could become one of > 2500 contributors to the project this way and give back for the free software.
Now that you know the issue and solutio, I guess you know best where you would look for this information so maybe proposing a PR in the way that would prevent users like you from running into the same problem is a good idea ?
Also I think it is part of the communication to have such issues that describe problem and solutions - hopefully guiding users to find a solution quickly rather than opening more duplicates (and start using constraints for the future as well).
Unfortunately, there are good reasons that 'pip install' without constraints might sometimes lead to problems like that triggered by external dependencies introducing breaking changes and while we are continuously monitoring and updating our specifications, the only way to get reproducible installs is by using constraints.
Also now that we know we have a breaking change we introduced limit and we work on fixing the incompatibilities in V2 of Pydantic for 2.6.3. So this is a temporary issue only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jens-scheffler-bosch closed issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "jens-scheffler-bosch (via GitHub)" <gi...@apache.org>.
jens-scheffler-bosch closed issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = <class 'str'>`.
URL: https://github.com/apache/airflow/issues/32332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jens-scheffler-bosch commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "jens-scheffler-bosch (via GitHub)" <gi...@apache.org>.
jens-scheffler-bosch commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1619002259
Duplicate of #32301
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "proofrock (via GitHub)" <gi...@apache.org>.
proofrock commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1620329508
I'll do that, @potiuk. Thanks for being such a nice project and people, it's really meaningful the way you treated me in this case, even if it was my fault (in good faith, of course). I appreciate it a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618881242
I see your Pydantic version is 2.0.0, can u try with a lower version? For more info see this - #32311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Adaverse commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "Adaverse (via GitHub)" <gi...@apache.org>.
Adaverse commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618892068
This will help - https://github.com/apache/airflow/issues/32316#issuecomment-1616703896
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618579236
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "proofrock (via GitHub)" <gi...@apache.org>.
proofrock commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1619623751
> > This will help - [#32316 (comment)](https://github.com/apache/airflow/issues/32316#issuecomment-1616703896)
>
> Thanks! I will try tomorrow morning (CET) and report back.
@Adaverse it works. Of course. Unfortunately - and it's my bad! There are some "wrong" guides out there, and they are considerably simpler to follow that the official guide, for an user who is not versed in python. Maybe it could be written on the home page or somewhere that some "alternative" methods don't work, so that you receive less duplicated issues. One follows the path of least resistance, so to speak. I will file a PR to the docs as soon as I test more.
But, again, my bad. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1620342707
It's nice to hear warm words like that :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "proofrock (via GitHub)" <gi...@apache.org>.
proofrock commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618885069
Hello and thanks for answering. How do I do that? I am not too familiar with python, all I can see is a pip command that installs a predefined dependency set (that includes that version of pydantic, while last thursday it did not). How can I "fine tune" the dependency? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] proofrock commented on issue #32332: Installation: pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `dag_id = `.
Posted by "proofrock (via GitHub)" <gi...@apache.org>.
proofrock commented on issue #32332:
URL: https://github.com/apache/airflow/issues/32332#issuecomment-1618898184
> This will help - https://github.com/apache/airflow/issues/32316#issuecomment-1616703896
Thanks! I will try tomorrow morning (CET) and report back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org